mir/kea
2
0
Fork 0
mirror of https://gitlab.isc.org/isc-projects/kea synced 2025-08-22 09:57:41 +00:00
Code Issues Releases Wiki Activity
kea/hammer.py

3221 lines
134 KiB
Python
Raw Normal View History

initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
#!/usr/bin/env python3
hammer: added copyright header
2019-01-10 17:48:15 +01:00
[#3738] Do not update repos on FreeBSD in hammer.py
2025-01-30 14:01:43 +02:00
# Copyright (C) 2018-2025 Internet Systems Consortium, Inc. ("ISC")
hammer: added copyright header
2019-01-10 17:48:15 +01:00
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
[#2350] hammer: install freeradius on prepare-system radius
2022-03-10 09:58:39 +02:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
# pylint: disable=broad-exception-caught
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
"""Hammer - Kea development environment management tool."""
hammer: added copyright header
2019-01-10 17:48:15 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
from __future__ import print_function
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
import os
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
import random
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
import re
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
import sys
import glob
import time
hammer: fixed conflict when hammer is used from 2 repo folders on the same computer, added jobs switch for forcing number of parallel compilation processes
2019-01-22 11:59:25 +01:00
import json
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
import logging
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
import datetime
hammer: fixed conflict when hammer is used from 2 repo folders on the same computer, added jobs switch for forcing number of parallel compilation processes
2019-01-22 11:59:25 +01:00
import platform
import binascii
import argparse
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
import textwrap
import functools
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
import multiprocessing
Fixed adding vagrant user to abuild group on Alpine
2020-10-20 14:58:03 +02:00
import grp
import pwd
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
import getpass
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
import urllib.request
from urllib.parse import urljoin
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
# [B404:blacklist] Consider possible security implications associated with subprocess module.
import subprocess # nosec B404
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
# Issue: [B405:blacklist] Using xml.etree.ElementTree to parse untrusted XML data is known to be vulnerable to XML
# attacks. Replace xml.etree.ElementTree with the equivalent defusedxml package, or make sure
# defusedxml.defuse_stdlib() is called.
import xml.etree.ElementTree as ET # nosec B405
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#2969] update supported-systems list in hammer.py
2023-10-10 18:51:23 +03:00
# SYSTEMS = {
[#3307] release changes
2024-03-22 15:55:27 +00:00
# 'os': {
# 'version': True if supported else False,
[#2969] update supported-systems list in hammer.py
2023-10-10 18:51:23 +03:00
# ...
[#3307] release changes
2024-03-22 15:55:27 +00:00
# },
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
# ...
# }
[#3307] release changes
2024-03-22 15:55:27 +00:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
SYSTEMS = {
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
'fedora': {
'27': False,
'28': False,
'29': False,
'30': False,
'31': False,
'32': False,
'33': False,
'34': False,
'35': False,
'36': False,
[#3307] release changes
2024-03-22 15:55:27 +00:00
'37': False,
[#3711] Update supported systems in hammer.py
2025-01-31 12:49:09 +02:00
'38': False,
'39': False,
[#3393] add fedora 40 to hammer
2024-05-13 14:41:05 +00:00
'40': True,
[#3664] add alpine 3.21 to hammer
2024-12-18 14:24:57 +00:00
'41': True,
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
},
'centos': {
'7': False,
'8': False,
'9': False,
},
'rhel': {
'8': True,
'9': True,
},
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
'rocky': {
'9': True,
},
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
'ubuntu': {
'16.04': False,
[#3711] Update supported systems in hammer.py
2025-01-31 12:49:09 +02:00
'18.04': False,
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
'18.10': False,
'19.04': False,
'19.10': False,
'20.04': True,
'20.10': False,
'21.04': False,
'22.04': True,
[#3382] add ubuntu 24.04 to hammer
2024-05-13 08:09:03 +00:00
'24.04': True,
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
},
'debian': {
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
'8': False,
'9': False,
[#3711] Update supported systems in hammer.py
2025-01-31 12:49:09 +02:00
'10': False,
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
'11': True,
'12': True,
},
'freebsd': {
'11.2': False,
'11.4': False,
'12.0': False,
'12.1': False,
[#3711] Update supported systems in hammer.py
2025-01-31 12:49:09 +02:00
'13.0': False,
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
'14.0': True,
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
},
'alpine': {
'3.10': False,
'3.11': False,
'3.12': False,
'3.13': False,
'3.14': False,
'3.15': False,
[#3711] Update supported systems in hammer.py
2025-01-31 12:49:09 +02:00
'3.16': False,
'3.17': False,
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
'3.18': True,
'3.19': True,
[#3445] add alpine 3.20 to hammer
2024-06-03 12:01:52 +00:00
'3.20': True,
[#3664] add alpine 3.21 to hammer
2024-12-18 14:24:57 +00:00
'3.21': True,
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
},
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
'arch': {},
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
}
IMAGE_TEMPLATES = {
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# fedora
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'fedora-27-lxc': {'bare': 'lxc-fedora-27', 'kea': 'godfryd/kea-fedora-27'},
'fedora-27-virtualbox': {'bare': 'generic/fedora27', 'kea': 'godfryd/kea-fedora-27'},
fixed bringing up lxc container when there is conflicting one present
2019-04-14 07:26:50 +02:00
'fedora-28-lxc': {'bare': 'godfryd/lxc-fedora-28', 'kea': 'godfryd/kea-fedora-28'},
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'fedora-28-virtualbox': {'bare': 'generic/fedora28', 'kea': 'godfryd/kea-fedora-28'},
'fedora-29-lxc': {'bare': 'godfryd/lxc-fedora-29', 'kea': 'godfryd/kea-fedora-29'},
'fedora-29-virtualbox': {'bare': 'generic/fedora29', 'kea': 'godfryd/kea-fedora-29'},
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
'fedora-30-lxc': {'bare': 'godfryd/lxc-fedora-30', 'kea': 'godfryd/kea-fedora-30'},
'fedora-30-virtualbox': {'bare': 'generic/fedora30', 'kea': 'godfryd/kea-fedora-30'},
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
'fedora-31-lxc': {'bare': 'isc/lxc-fedora-31', 'kea': 'isc/kea-fedora-31'},
'fedora-31-virtualbox': {'bare': 'isc/vbox-fedora-31', 'kea': 'isc/kea-fedora-31'},
[#1229] hammer: added support for building kea on fedora 32 and ubuntu 20.04
2020-05-12 11:39:17 +02:00
'fedora-32-lxc': {'bare': 'isc/lxc-fedora-32', 'kea': 'isc/kea-fedora-32'},
[#1527] added support for fedora 33 and ubunut 20.10
2021-01-19 19:09:45 +01:00
'fedora-33-lxc': {'bare': 'isc/lxc-fedora-33', 'kea': 'isc/kea-fedora-33'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
'fedora-34-lxc': {'bare': 'isc/lxc-fedora-34', 'kea': 'isc/kea-fedora-34'},
[#2410] fedora 35 and 35 support
2022-05-18 09:22:15 +02:00
'fedora-35-lxc': {'bare': 'isc/lxc-fedora-35', 'kea': 'isc/kea-fedora-35'},
'fedora-36-lxc': {'bare': 'isc/lxc-fedora-36', 'kea': 'isc/kea-fedora-36'},
[#2935] Add Fedora 37 support to hammer
2023-06-20 16:07:57 +00:00
'fedora-37-lxc': {'bare': 'isc/lxc-fedora-37', 'kea': 'isc/kea-fedora-37'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# centos
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
'centos-7-lxc': {'bare': 'isc/lxc-centos-7', 'kea': 'isc/kea-centos-7'},
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'centos-7-virtualbox': {'bare': 'generic/centos7', 'kea': 'godfryd/kea-centos-7'},
[#934,!531] kea-centos-8 for virtualbox and lxc uploaded and available
2019-10-14 22:59:54 +07:00
'centos-8-lxc': {'bare': 'isc/lxc-centos-8', 'kea': 'isc/kea-centos-8'},
'centos-8-virtualbox': {'bare': 'generic/centos8', 'kea': 'isc/kea-centos-8'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# rhel
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
'rhel-8-virtualbox': {'bare': 'generic/rhel8', 'kea': 'generic/rhel8'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# ubuntu
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'ubuntu-16.04-lxc': {'bare': 'godfryd/lxc-ubuntu-16.04', 'kea': 'godfryd/kea-ubuntu-16.04'},
'ubuntu-16.04-virtualbox': {'bare': 'ubuntu/xenial64', 'kea': 'godfryd/kea-ubuntu-16.04'},
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
'ubuntu-18.04-lxc': {'bare': 'isc/lxc-ubuntu-18.04', 'kea': 'isc/kea-ubuntu-18.04'},
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'ubuntu-18.04-virtualbox': {'bare': 'ubuntu/bionic64', 'kea': 'godfryd/kea-ubuntu-18.04'},
'ubuntu-18.10-lxc': {'bare': 'godfryd/lxc-ubuntu-18.10', 'kea': 'godfryd/kea-ubuntu-18.10'},
'ubuntu-18.10-virtualbox': {'bare': 'ubuntu/cosmic64', 'kea': 'godfryd/kea-ubuntu-18.10'},
hammer: added support for ubuntu 19.04
2019-05-10 16:07:23 +02:00
'ubuntu-19.04-lxc': {'bare': 'godfryd/lxc-ubuntu-19.04', 'kea': 'godfryd/kea-ubuntu-19.04'},
'ubuntu-19.04-virtualbox': {'bare': 'ubuntu/disco64', 'kea': 'godfryd/kea-ubuntu-19.04'},
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
'ubuntu-19.10-lxc': {'bare': 'isc/lxc-ubuntu-19.10', 'kea': 'isc/kea-ubuntu-19.10'},
'ubuntu-19.10-virtualbox': {'bare': 'generic/ubuntu1910', 'kea': 'isc/kea-ubuntu-19.10'},
[#1229] hammer: added support for building kea on fedora 32 and ubuntu 20.04
2020-05-12 11:39:17 +02:00
'ubuntu-20.04-lxc': {'bare': 'isc/lxc-ubuntu-20.04', 'kea': 'isc/kea-ubuntu-20.04'},
[#1527] added support for fedora 33 and ubunut 20.10
2021-01-19 19:09:45 +01:00
'ubuntu-20.10-lxc': {'bare': 'isc/lxc-ubuntu-20.10', 'kea': 'isc/kea-ubuntu-20.10'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
'ubuntu-21.04-lxc': {'bare': 'isc/lxc-ubuntu-21.04', 'kea': 'isc/kea-ubuntu-21.04'},
[#2433] add support for ubuntu 22.04
2022-06-08 17:46:33 +02:00
'ubuntu-22.04-lxc': {'bare': 'isc/lxc-ubuntu-22.04', 'kea': 'isc/lxc-ubuntu-22.04'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# debian
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'debian-8-lxc': {'bare': 'godfryd/lxc-debian-8', 'kea': 'godfryd/kea-debian-8'},
'debian-8-virtualbox': {'bare': 'debian/jessie64', 'kea': 'godfryd/kea-debian-8'},
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
'debian-9-lxc': {'bare': 'isc/lxc-debian-9', 'kea': 'isc/kea-debian-9'},
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'debian-9-virtualbox': {'bare': 'debian/stretch64', 'kea': 'godfryd/kea-debian-9'},
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
'debian-10-lxc': {'bare': 'isc/lxc-debian-10', 'kea': 'isc/kea-debian-10'},
hammer: added support for debian-10, some other various fixes
2019-06-24 21:58:48 +02:00
'debian-10-virtualbox': {'bare': 'debian/buster64', 'kea': 'godfryd/kea-debian-10'},
[#2042] enabled debian 11
2021-11-12 01:07:13 +01:00
'debian-11-lxc': {'bare': 'isc/lxc-debian-11', 'kea': 'isc/kea-debian-11'},
[#2916] hammer changes for debian 12
2023-06-15 13:23:44 +00:00
'debian-12-lxc': {'bare': 'isc/lxc-debian-12', 'kea': 'isc/kea-debian-12'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# freebsd
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
'freebsd-11.2-virtualbox': {'bare': 'generic/freebsd11', 'kea': 'godfryd/kea-freebsd-11.2'},
'freebsd-12.0-virtualbox': {'bare': 'generic/freebsd12', 'kea': 'godfryd/kea-freebsd-12.0'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
'freebsd-13.0-virtualbox': {'bare': 'isc/vbox-freebsd-13.0', 'kea': 'isc/kea-freebsd-13.0'},
# alpine
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
'alpine-3.10-lxc': {'bare': 'godfryd/lxc-alpine-3.10', 'kea': 'godfryd/kea-alpine-3.10'},
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
'alpine-3.11-lxc': {'bare': 'isc/lxc-alpine-3.11', 'kea': 'isc/kea-alpine-3.11'},
[#1429] hammer: added support for building Kea on Alpine 3.12
2020-09-23 17:50:08 +02:00
'alpine-3.12-lxc': {'bare': 'isc/lxc-alpine-3.12', 'kea': 'isc/kea-alpine-3.12'},
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
'alpine-3.13-lxc': {'bare': 'isc/lxc-alpine-3.13', 'kea': 'isc/kea-alpine-3.13'},
Hammer: Add support for Alpine 3.14 and 3.15
2022-02-01 05:19:12 -08:00
'alpine-3.14-lxc': {'bare': 'isc/lxc-alpine-3.14', 'kea': 'isc/kea-alpine-3.14'},
'alpine-3.15-lxc': {'bare': 'isc/lxc-alpine-3.15', 'kea': 'isc/kea-alpine-3.15'},
[#2491] Add hammer support for alpine 3.16
2022-06-29 14:23:25 -07:00
'alpine-3.16-lxc': {'bare': 'isc/lxc-alpine-3.16', 'kea': 'isc/kea-alpine-3.16'},
[#2803] enable alpine 3.17 in hammer
2023-03-23 12:54:34 +00:00
'alpine-3.17-lxc': {'bare': 'isc/lxc-alpine-3.17', 'kea': 'isc/kea-alpine-3.17'},
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
}
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
# NOTES
# ** Alpine **
# 1. Extracting rootfs is failing:
[#1760] spell checks
2021-03-18 10:22:04 +02:00
# It requires commenting out checking if rootfs has been extracted as it checks for file /bin/true which is a link.
[#1429] hammer: added support for building Kea on Alpine 3.12
2020-09-23 17:50:08 +02:00
# Comment out in ~/.vagrant.d/gems/2.X.Y/gems/vagrant-lxc-1.4.3/scripts/lxc-template near 'Failed to extract rootfs'
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
LXC_VAGRANTFILE_TPL = """# -*- mode: ruby -*-
# vi: set ft=ruby :
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
ENV["LC_ALL"] = "C"
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
Vagrant.configure("2") do |config|
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
{hostname}
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
config.vm.box = "{image_tpl}"
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
{box_version}
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
config.vm.provider "lxc" do |lxc|
lxc.container_name = "{name}"
hammer: fixed handling lxc rootfs path
2019-01-22 13:05:20 +01:00
lxc.customize 'rootfs.path', "/var/lib/lxc/{name}/rootfs"
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
end
config.vm.synced_folder '.', '/vagrant', disabled: true
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
config.vm.synced_folder '{ccache_dir}', '/ccache'
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
end
"""
VBOX_VAGRANTFILE_TPL = """# -*- mode: ruby -*-
# vi: set ft=ruby :
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
ENV["LC_ALL"] = "C"
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
Vagrant.configure("2") do |config|
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
config.vm.hostname = "{name}"
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
config.vm.box = "{image_tpl}"
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
{box_version}
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
config.vm.provider "virtualbox" do |v|
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
v.name = "{name}"
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
v.memory = 8192
nproc = Etc.nprocessors
if nproc > 8
nproc -= 2
elsif nproc > 1
nproc -= 1
end
v.cpus = nproc
end
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
config.vm.synced_folder '.', '/vagrant', disabled: true
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
end
"""
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
RECOMMENDED_VAGRANT_VERSION = '2.2.16'
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
log = logging.getLogger()
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
def red(txt):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Return colorized (if the terminal supports it) or plain text."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if sys.stdout.isatty():
return '\033[1;31m%s\033[0;0m' % txt
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
return txt
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
def green(txt):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Return colorized (if the terminal supports it) or plain text."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if sys.stdout.isatty():
return '\033[0;32m%s\033[0;0m' % txt
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
return txt
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
def blue(txt):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Return colorized (if the terminal supports it) or plain text."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if sys.stdout.isatty():
return '\033[0;34m%s\033[0;0m' % txt
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
return txt
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def get_system_revision():
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Return tuple containing system name and its revision."""
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
system = platform.system()
if system == 'Linux':
[#1229] hammer: added support for building kea on fedora 32 and ubuntu 20.04
2020-05-12 11:39:17 +02:00
system, revision = None, None
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
if not os.path.exists('/etc/os-release'):
raise UnexpectedError('/etc/os-release does not exist. Cannot determine system or its revision.')
vals = {}
with open('/etc/os-release', encoding='utf-8') as f:
for line in f.readlines():
if '=' in line:
key, val = line.split('=', 1)
vals[key.strip()] = val.strip().replace('"', '')
for i in ['ID', 'ID_LIKE']:
if i in vals:
system_candidates = vals[i].strip('"').split()
for system_candidate in system_candidates:
if system_candidate in SYSTEMS:
system = system_candidate
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
break
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
else:
continue
break
if system is None:
raise UnexpectedError('cannot determine system')
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
for i in ['VERSION_ID', 'BUILD_ID']:
if i in vals:
revision = vals[i]
break
if revision is None:
raise UnexpectedError('cannot determine revision')
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
if system in ['alpine', 'rhel', 'rocky']:
revision = revision.rsplit('.', 1)[0]
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif system == 'FreeBSD':
system = system.lower()
revision = platform.release()
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
if '"' in revision:
revision = revision.replace('"', '')
Fix the issue with CentOS 8 not being recognized
2020-12-03 00:14:40 +01:00
if '"' in system:
system = system.replace('"', '')
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
system = system.lower()
return system, revision
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
class ExecutionError(Exception):
[#372,!181] Added many missing comments
2019-01-30 20:44:35 +01:00
"""Exception thrown when execution encountered an error."""
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
class UnexpectedError(Exception):
"""Exception thrown when an unexpected error occurred that hammer does not know how to recover from."""
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
def execute(cmd, timeout=60, cwd=None, env=None, raise_error=True, dry_run=False, log_file_path=None,
quiet=False, check_times=False, capture=False, interactive=False, attempts=1,
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
sleep_time_after_attempt=None, super_quiet=False):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Execute a command in shell.
:param str cmd: a command to be executed
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
:param int timeout: timeout in number of seconds, after that time the command is terminated
but only if check_times is True
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
:param str cwd: current working directory for the command
:param dict env: dictionary with environment variables
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
:param bool raise_error: if False then in case of error exception is not raised,
default: True ie exception is raise
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
:param bool dry_run: if True then the command is not executed
:param str log_file_path: if provided then all traces from the command are stored in indicated file
:param bool quiet: if True then the command's traces are not printed to stdout
:param bool check_times: if True then timeout is taken into account
:param bool capture: if True then the command's traces are captured and returned by the function
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
:param bool interactive: if True then stdin and stdout are not redirected, traces handling is disabled,
used for e.g. SSH
[#1262] hammer: hardened checking installed packages
2020-05-29 15:25:16 +02:00
:param int attempts: number of attempts to run the command if it fails
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
:param int sleep_time_after_attempt: number of seconds to sleep before taking next attempt
[#2539] mostly cosmetical changes to hammer.py
2022-09-05 19:32:22 +03:00
:param bool super_quiet: if True, set quiet to True and don't log command
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
if super_quiet:
quiet = True
[#2657] replace ~/ with absolute path
2022-11-30 12:29:34 +01:00
if cwd and "~/" in cwd:
# replace relative home directory
[#2696] hammer.py: replace ~ with os.environ['HOME'] On rare occasion, the previously-used execute() would return empty output, which would cause the path to change, often resulting in a 'no such file or directory' error.
2023-01-25 22:38:59 +02:00
cwd = cwd.replace('~', os.environ['HOME'])
[#2657] changed sequence of command logging
2022-11-30 12:42:10 +01:00
if not super_quiet:
log.info('>>>>> Executing %s in %s', cmd, cwd if cwd else os.getcwd())
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if not check_times:
timeout = None
if dry_run:
return 0
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
if 'sudo' in cmd and env:
# if sudo is used and env is overridden then to preserve env add -E to sudo
cmd = cmd.replace('sudo', 'sudo -E')
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
if log_file_path:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open(log_file_path, "wb", encoding='utf-8') as file:
log_file = file.read()
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
for attempt in range(attempts):
if interactive:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
# Issue: [B602:subprocess_popen_with_shell_equals_true] subprocess call with shell=True identified,
# security issue.
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with subprocess.Popen(cmd, cwd=cwd, env=env, shell=True) as pipe: # nosec: B602
pipe.communicate()
exitcode = pipe.returncode
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
else:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
# Issue: [B602:subprocess_popen_with_shell_equals_true] subprocess call with shell=True identified,
# security issue.
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with subprocess.Popen(cmd, cwd=cwd, env=env, shell=True, # nosec: B602
stdout=subprocess.PIPE, stderr=subprocess.STDOUT) as pipe:
try:
[#3287] address review
2024-06-17 18:26:05 +03:00
if timeout is not None:
pipe.wait(timeout)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
stdout, _ = pipe.communicate()
except subprocess.TimeoutExpired as e:
pipe.kill()
stdout2, _ = pipe.communicate()
stdout += stdout2
raise ExecutionError(f'Execution timeout: {e}, cmd: {cmd}') from e
exitcode = pipe.returncode
if capture:
output = stdout.decode('utf-8')
if not quiet:
print(stdout.decode('utf-8'))
if log_file_path is not None:
log_file.write(stdout)
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
if exitcode == 0:
break
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
if attempt < attempts - 1:
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
txt = 'command failed, retry, attempt %d/%d' % (attempt, attempts)
if log_file_path:
txt_to_file = '\n\n[HAMMER] %s\n\n\n' % txt
log_file.write(txt_to_file.encode('ascii'))
log.info(txt)
if sleep_time_after_attempt:
time.sleep(sleep_time_after_attempt)
if log_file_path:
log_file.close()
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if exitcode != 0 and raise_error:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if capture and quiet:
log.error(output)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
raise ExecutionError("The command return non-zero exitcode %s, cmd: '%s'" % (exitcode, cmd))
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
if capture:
return exitcode, output
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
return exitcode
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#3559] hammer.py: pkill mariadb_safe before restarting Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-27 17:36:08 +03:00
def wait_for_process_to_start(process_name):
for _ in range(10):
exit_code = execute(f'sudo pidof {process_name}', raise_error=False)
if exit_code == 0:
# Process is there.
break
time.sleep(1)
[#3547] hammer.py: wait for start-stop-daemon/mysqld_safe to exit Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-22 10:23:46 +03:00
def wait_for_process_to_exit(process_name):
for _ in range(100):
exit_code = execute(f'sudo pidof {process_name}', raise_error=False)
if exit_code != 0:
# Process exited or there was no process to begin with.
break
time.sleep(1)
[#2510] hammer.py: export the name of FreeRADIUS client packages used while building packages
2022-07-26 22:01:31 +03:00
def _append_to_file(file_name, line):
with open(file_name, encoding='utf-8', mode='a') as f:
f.write(line + '\n')
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
def _prepare_installed_packages_cache_for_debs():
pkg_cache = {}
_, out = execute("dpkg -l", timeout=15, capture=True, quiet=True)
for line in out.splitlines():
line = line.strip()
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
m = re.search(r'^([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+(.+)', line)
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if not m:
continue
status, name, version, arch, descr = m.groups()
name = name.split(':')[0]
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_cache[name] = {
'status': status,
'version': version,
'arch': arch,
'descr': descr,
}
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
return pkg_cache
def _prepare_installed_packages_cache_for_rpms():
pkg_cache = {}
_, out = execute("rpm -qa --qf '%{NAME}\\n'", timeout=15, capture=True, quiet=True)
for line in out.splitlines():
name = line.strip()
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_cache[name] = {'status': 'ii'}
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
return pkg_cache
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
def _prepare_installed_packages_cache_for_alpine():
pkg_cache = {}
_, out = execute("apk list -I\\n'", timeout=15, capture=True, quiet=True)
for line in out.splitlines():
name = line.strip()
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_cache[name] = {'status': 'ii'}
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
return pkg_cache
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
def install_pkgs(pkgs, timeout=60, env=None, check_times=False, pkg_cache=None):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Install native packages in a system.
[#372,!181] Added many missing comments
2019-01-30 20:44:35 +01:00
:param dict pkgs: specifies a list of packages to be installed
:param int timeout: timeout in number of seconds, after that time the command
is terminated but only if check_times is True
:param dict env: dictionary with environment variables (optional)
:param bool check_times: specifies if timeouts should be enabled (optional)
"""
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
system, revision = get_system_revision()
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if not isinstance(pkgs, list):
pkgs = pkgs.split()
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
if pkg_cache is None:
pkg_cache = {}
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
# prepare cache if needed
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
if not pkg_cache and system in ['centos', 'rhel', 'fedora', 'debian', 'ubuntu',
'rocky']: # , 'alpine']: # TODO: complete caching support for alpine
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if system in ['centos', 'rhel', 'fedora', 'rocky']:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
pkg_cache.update(_prepare_installed_packages_cache_for_rpms())
elif system in ['debian', 'ubuntu']:
pkg_cache.update(_prepare_installed_packages_cache_for_debs())
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
elif system in ['alpine']:
pkg_cache.update(_prepare_installed_packages_cache_for_alpine())
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
# check if packages actually need to be installed
if pkg_cache:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
pkgs_to_install = []
[#1808] added freeradius-client version for fedora 33 and other minor fixes
2021-04-15 10:28:04 +02:00
pkgs_installed = []
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
for pkg in pkgs:
if pkg not in pkg_cache or pkg_cache[pkg]['status'] != 'ii':
pkgs_to_install.append(pkg)
[#1808] added freeradius-client version for fedora 33 and other minor fixes
2021-04-15 10:28:04 +02:00
else:
pkgs_installed.append(pkg)
if pkgs_installed:
log.info('packages already installed: %s', ', '.join(pkgs_installed))
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
pkgs = pkgs_to_install
if not pkgs:
log.info('all packages already installed')
return
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if system in ['centos', 'fedora', 'rhel', 'rocky']:
[#2539] hammer.py: install dnf on CentOS 7 and RHEL 7 Fixes the following error that appeared recently on CentOS 7. Error: Not tolerating missing names on install, stopping.
2022-09-05 22:30:17 +03:00
if system in ['centos', 'rhel'] and revision == '7':
execute('sudo yum install -y dnf')
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
cmd = 'sudo dnf -y install'
elif system in ['debian', 'ubuntu']:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
# prepare the command for ubuntu/debian
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
if not env:
env = os.environ.copy()
env['DEBIAN_FRONTEND'] = 'noninteractive'
cmd = 'sudo apt install --no-install-recommends -y'
elif system == 'freebsd':
[#3738] Do not update repos on FreeBSD in hammer.py
2025-01-30 14:01:43 +02:00
cmd = 'sudo pkg install --no-repo-update --yes'
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
elif system == 'alpine':
cmd = 'sudo apk add'
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
elif system == 'arch':
cmd = 'sudo pacman -S --needed --noconfirm --overwrite \'*\''
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
else:
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
raise NotImplementedError('no implementation for %s' % system)
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
pkgs = ' '.join(pkgs)
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
cmd += ' ' + pkgs
execute(cmd, timeout=timeout, env=env, check_times=check_times, attempts=3, sleep_time_after_attempt=10)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#2112] hammer: better error message about unsupported system
2021-11-03 18:03:03 +02:00
def get_image_template(key, variant):
if key not in IMAGE_TEMPLATES:
[#2112] solve python syntax error in hammer
2021-11-19 19:17:36 +02:00
print('ERROR: Image {} is not available.'.format(key), file=sys.stderr)
[#2112] hammer: better error message about unsupported system
2021-11-03 18:03:03 +02:00
sys.exit(1)
if variant not in IMAGE_TEMPLATES[key]:
[#2112] solve python syntax error in hammer
2021-11-19 19:17:36 +02:00
print('ERROR: Variant {} is not available for image {}.'.format(variant, key), file=sys.stderr)
[#2112] hammer: better error message about unsupported system
2021-11-03 18:03:03 +02:00
sys.exit(1)
return IMAGE_TEMPLATES[key][variant]
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
def _get_full_repo_url(repository_url, system, revision):
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
if not repository_url:
return None
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
repo_name = 'kea-%s-%s' % (system, revision)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
repo_url = urljoin(repository_url, 'repository')
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
repo_url += '/%s/' % repo_name
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
return repo_url
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
class VagrantEnv():
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Helper class that makes interacting with Vagrant easier.
It creates Vagrantfile according to specified system. It exposes basic Vagrant functions
[#1760] spell checks
2021-03-18 10:22:04 +02:00
like up, upload, destroy, ssh. It also provides more complex function for preparing system
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
for Kea build and building Kea.
"""
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
def __init__(self, provider, system, revision, features, image_template_variant,
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
dry_run, quiet=False, check_times=False, ccache_dir=None):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""VagrantEnv initializer.
:param str provider: indicate backend type: virtualbox or lxc
:param str system: name of the system eg. ubuntu
:param str revision: revision of the system e.g. 18.04
:param list features: list of requested features
:param str image_template_variant: variant of images' templates: bare or kea
:param bool dry_run: if False then system commands are not really executed
:param bool quiet: if True then commands will not trace to stdout
:param bool check_times: if True then commands will be terminated after given timeout
"""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
self.provider = provider
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
self.system = system
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
self.revision = revision
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
self.features = features
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
self.dry_run = dry_run
self.quiet = quiet
self.check_times = check_times
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# set properly later
self.features_arg = None
self.nofeatures_arg = None
self.python = None
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
self.key = key = "%s-%s-%s" % (system, revision, provider)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
self.image_tpl = get_image_template(key, image_template_variant)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
self.repo_dir = os.getcwd()
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
sys_dir = "%s-%s" % (system, revision)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if provider == "virtualbox":
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
self.vagrant_dir = os.path.join(self.repo_dir, 'hammer', sys_dir, 'vbox')
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif provider == "lxc":
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
self.vagrant_dir = os.path.join(self.repo_dir, 'hammer', sys_dir, 'lxc')
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if ccache_dir is None:
self.ccache_dir = '/'
self.ccache_enabled = False
else:
self.ccache_dir = ccache_dir
self.ccache_enabled = True
self.init_files()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
def init_files(self):
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if not os.path.exists(self.vagrant_dir):
os.makedirs(self.vagrant_dir)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
vagrantfile_path = os.path.join(self.vagrant_dir, "Vagrantfile")
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: fixed conflict when hammer is used from 2 repo folders on the same computer, added jobs switch for forcing number of parallel compilation processes
2019-01-22 11:59:25 +01:00
crc = binascii.crc32(self.vagrant_dir.encode())
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
self.name = "hmr-%s-%s-kea-srv-%08d" % (self.system, self.revision.replace('.', '-'), crc)
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if '/' in self.image_tpl:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
self.latest_version = self._get_latest_cloud_version()
box_version = 'config.vm.box_version = "%s"' % self.latest_version
else:
self.latest_version = None
box_version = ""
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
# alpine has a problem with setting hostname so skip it
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if self.system == 'alpine':
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
hostname = ''
else:
hostname = 'config.vm.hostname = "%s"' % self.name
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if self.provider == "virtualbox":
vagrantfile_tpl = VBOX_VAGRANTFILE_TPL
elif self.provider == "lxc":
vagrantfile_tpl = LXC_VAGRANTFILE_TPL
vagrantfile = vagrantfile_tpl.format(image_tpl=self.image_tpl,
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
name=self.name,
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
ccache_dir=self.ccache_dir,
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
box_version=box_version,
hostname=hostname)
hammer: fixed conflict when hammer is used from 2 repo folders on the same computer, added jobs switch for forcing number of parallel compilation processes
2019-01-22 11:59:25 +01:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open(vagrantfile_path, "w", encoding='utf-8') as f:
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
f.write(vagrantfile)
hammer: fixed conflict when hammer is used from 2 repo folders on the same computer, added jobs switch for forcing number of parallel compilation processes
2019-01-22 11:59:25 +01:00
log.info('Prepared vagrant system %s in %s', self.name, self.vagrant_dir)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def up(self):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Do Vagrant up."""
fixed bringing up lxc container when there is conflicting one present
2019-04-14 07:26:50 +02:00
exitcode, out = execute("vagrant up --no-provision --provider %s" % self.provider,
cwd=self.vagrant_dir, timeout=15 * 60, dry_run=self.dry_run,
capture=True, raise_error=False)
if exitcode != 0:
if 'There is container on your system' in out and 'lxc-destroy' in out:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
m = re.search(r'`lxc-destroy.*?`', out)
fixed bringing up lxc container when there is conflicting one present
2019-04-14 07:26:50 +02:00
if m:
# destroy some old container
cmd = m.group(0)[1:-1]
cmd = 'sudo ' + cmd + ' -f'
execute(cmd, timeout=60)
# try again spinning up new
execute("vagrant up --no-provision --provider %s" % self.provider,
cwd=self.vagrant_dir, timeout=15 * 60, dry_run=self.dry_run)
return
raise ExecutionError('There is a problem with putting up a system')
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
def _get_cloud_meta(self, image_tpl=None):
if '/' not in self.image_tpl:
return {}
url = 'https://app.vagrantup.com/api/v1/box/' + (image_tpl if image_tpl else self.image_tpl)
try:
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
# Issue: [B310:blacklist] Audit url open for permitted schemes.
# Allowing use of file:/ or custom schemes is often unexpected.
# Reason for nosec: it is clearly a https link.
with urllib.request.urlopen(url) as response: # nosec B310
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
data = response.read()
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
except Exception as e:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.exception('ignored exception: %s', e)
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
return {}
data = json.loads(data)
return data
def _get_local_meta(self):
meta_file = os.path.join(self.vagrant_dir, '.vagrant/machines/default', self.provider, 'box_meta')
if not os.path.exists(meta_file):
return {}
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open(meta_file, encoding='utf-8') as f:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
data = f.read()
data = json.loads(data)
return data
def _get_latest_cloud_version(self, image_tpl=None):
cloud_meta = self._get_cloud_meta(image_tpl)
if not cloud_meta and 'versions' not in cloud_meta:
return 0
latest_version = 0
for ver in cloud_meta['versions']:
provider_found = False
for p in ver['providers']:
if p['name'] == self.provider:
provider_found = True
break
if provider_found:
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
try:
v = int(ver['number'])
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
except ValueError:
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
return ver['number']
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if v > latest_version:
latest_version = v
return latest_version
def get_status(self):
"""Return system status.
Status can be: 'not created', 'running', 'stopped', etc.
"""
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if not os.path.exists(self.vagrant_dir):
return "not created"
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
_, out = execute("vagrant status", cwd=self.vagrant_dir, timeout=15, capture=True, quiet=True)
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
m = re.search(r'default\s+(.+)\(', out)
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if not m:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
raise UnexpectedError('cannot get status in:\n%s' % out)
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
return m.group(1).strip()
def bring_up_latest_box(self):
if self.get_status() == 'running':
self.reload()
else:
self.up()
def reload(self):
"""Do Vagrant reload."""
execute("vagrant reload --no-provision --force",
cwd=self.vagrant_dir, timeout=15 * 60, dry_run=self.dry_run)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def package(self):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Package Vagrant system into Vagrant box."""
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
execute('vagrant halt', cwd=self.vagrant_dir, dry_run=self.dry_run, raise_error=False, attempts=3)
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
hammer: added support for debian-10, some other various fixes
2019-06-24 21:58:48 +02:00
box_path = os.path.join(self.vagrant_dir, 'kea-%s-%s.box' % (self.system, self.revision))
if os.path.exists(box_path):
os.unlink(box_path)
hammer: added support for ubuntu 19.04
2019-05-10 16:07:23 +02:00
hammer: added support for debian-10, some other various fixes
2019-06-24 21:58:48 +02:00
if self.provider == 'virtualbox':
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
cmd = "vagrant package --output %s" % box_path
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
execute(cmd, cwd=self.vagrant_dir, timeout=4 * 60, dry_run=self.dry_run)
elif self.provider == 'lxc':
lxc_box_dir = os.path.join(self.vagrant_dir, 'lxc-box')
if os.path.exists(lxc_box_dir):
execute('sudo rm -rf %s' % lxc_box_dir)
os.mkdir(lxc_box_dir)
lxc_container_path = os.path.join('/var/lib/lxc', self.name)
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
# add vagrant universal key to accepted keys
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
execute('sudo sh -c \'echo "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8ia'
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
'llvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ'
'6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTB'
'ckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6k'
'ivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmB'
'YSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYC'
'zRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key"'
'> %s/rootfs/home/vagrant/.ssh/authorized_keys\'' % lxc_container_path)
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
# reset machine-id
execute('sudo rm -f %s/rootfs/var/lib/dbus/machine-id' % lxc_container_path)
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
# execute('sudo truncate -s 0 %s/rootfs/etc/machine-id' % lxc_container_path)
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
execute('sudo rm -f %s/rootfs/etc/machine-id' % lxc_container_path)
# pack rootfs
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = 'sudo sh -c "'
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
cmd += 'cd %s '
cmd += '&& tar --numeric-owner --anchored --exclude=./rootfs/dev/log -czf %s/rootfs.tar.gz ./rootfs/*'
cmd += '"'
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
execute(cmd % (lxc_container_path, lxc_box_dir))
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
# copy lxc config from runtime container
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
execute('sudo cp %s/config %s/lxc-config' % (lxc_container_path, lxc_box_dir))
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
# remove mac address from eth0 - it should be dynamically assigned
execute("sudo sed -i '/lxc.net.0.hwaddr/d' %s/lxc-config" % lxc_box_dir)
# correct files ownership
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
execute('sudo chown `id -un`:`id -gn` *', cwd=lxc_box_dir)
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
# and other metadata
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open(os.path.join(lxc_box_dir, 'metadata.json'), 'w', encoding='utf-8') as f:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
now = datetime.datetime.now()
f.write('{\n')
f.write(' "provider": "lxc",\n')
f.write(' "version": "1.0.0",\n')
f.write(' "built-on": "%s"\n' % now.strftime('%c'))
f.write('}\n')
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
# pack vagrant box with metadata and config
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
execute('tar -czf %s ./*' % box_path, cwd=lxc_box_dir)
execute('sudo rm -rf %s' % lxc_box_dir)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
return box_path
def upload_to_cloud(self, box_path):
[#2112] hammer: better error message about unsupported system
2021-11-03 18:03:03 +02:00
image_tpl = get_image_template(self.key, 'kea')
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if '/' not in image_tpl:
return
latest_version = self._get_latest_cloud_version(image_tpl)
new_version = latest_version + 1
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
cmd = "vagrant cloud publish --no-private -f -r %s %s %s %s"
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
cmd = cmd % (image_tpl, new_version, self.provider, box_path)
execute(cmd, cwd=self.vagrant_dir, timeout=60 * 60)
hammer: hardened vagrant upload, tuned timeouts
2019-01-24 15:32:03 +01:00
def upload(self, src):
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
"""Upload src to Vagrant system, home folder."""
hammer: hardened vagrant upload, tuned timeouts
2019-01-24 15:32:03 +01:00
attempt = 4
while attempt > 0:
exitcode = execute('vagrant upload %s' % src, cwd=self.vagrant_dir, dry_run=self.dry_run, raise_error=False)
if exitcode == 0:
break
attempt -= 1
if exitcode != 0:
msg = 'cannot upload %s' % src
log.error(msg)
raise ExecutionError(msg)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
def run_build_and_test(self, tarball_path, jobs, pkg_version, pkg_isc_version, upload, repository_url):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Run build and unit tests inside Vagrant system."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if self.dry_run:
return 0, 0
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare tarball if needed and upload it to vagrant system
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if not tarball_path:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('mkdir -p ~/.hammer-tmp')
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
name_ver = 'kea-%s' % pkg_version
hammer: added building with radius
2019-01-17 16:41:03 +01:00
cmd = 'tar --transform "flags=r;s|^|%s/|" --exclude hammer ' % name_ver
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
cmd += ' --exclude "*~" --exclude .git --exclude .libs '
cmd += ' --exclude .deps --exclude \'*.o\' --exclude \'*.lo\' '
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
cmd += ' -zcf ~/.hammer-tmp/%s.tar.gz .' % name_ver
hammer: added building with radius
2019-01-17 16:41:03 +01:00
execute(cmd)
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
tarball_path = '~/.hammer-tmp/%s.tar.gz' % name_ver
hammer: hardened vagrant upload, tuned timeouts
2019-01-24 15:32:03 +01:00
self.upload(tarball_path)
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
log_file_path = os.path.join(self.vagrant_dir, 'build.log')
log.info('Build log file stored to %s', log_file_path)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
t0 = time.time()
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# run build command
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
bld_cmd = "{python} hammer.py build -p local {features} {nofeatures} {check_times} {ccache}"
bld_cmd += " {tarball} {jobs} {pkg_version} {pkg_isc_version} {repository_url}"
bld_cmd = bld_cmd.format(python=self.python,
features=self.features_arg,
nofeatures=self.nofeatures_arg,
check_times='-i' if self.check_times else '',
ccache='--ccache-dir /ccache' if self.ccache_enabled else '',
[#1808] added freeradius-client version for fedora 33 and other minor fixes
2021-04-15 10:28:04 +02:00
tarball='-t ~/%s.tar.gz' % name_ver,
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
jobs='-j %d' % jobs,
pkg_version='--pkg-version %s' % pkg_version,
pkg_isc_version='--pkg-isc-version %s' % pkg_isc_version,
repository_url=('--repository-url %s' % repository_url) if repository_url else '')
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
timeout = _calculate_build_timeout(self.features) + 5 * 60
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
# executes hammer.py inside LXC container
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
self.execute(bld_cmd, timeout=timeout, log_file_path=log_file_path, quiet=self.quiet) # timeout: 40 minutes
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
ssh_cfg_path = self.dump_ssh_config()
if 'native-pkg' in self.features:
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
pkgs_dir = os.path.join(self.vagrant_dir, 'pkgs')
if os.path.exists(pkgs_dir):
execute('rm -rf %s' % pkgs_dir)
os.makedirs(pkgs_dir)
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
# copy results of _build_native_pkg
[#1873] hammer: dropped support for python3 in building kea packages
2021-05-17 08:57:57 +02:00
execute('scp -F %s -r default:~/kea-pkg/* .' % ssh_cfg_path, cwd=pkgs_dir)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
if upload:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
repo_url = _get_full_repo_url(repository_url, self.system, self.revision)
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
if repo_url is None:
raise ValueError('repo_url is None')
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
upload_cmd = 'curl -v --netrc -f'
if self.system in ['ubuntu', 'debian']:
upload_cmd += ' -X POST -H "Content-Type: multipart/form-data" --data-binary "@%s" '
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
file_ext = 'deb' # include both '.deb' and '.ddeb' files
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
elif self.system in ['fedora', 'centos', 'rhel', 'rocky']:
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
upload_cmd += ' --upload-file %s '
file_ext = '.rpm'
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
elif self.system == 'alpine':
upload_cmd += ' --upload-file %s '
file_ext = ''
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
_, arch = self.execute('arch', raise_error=False, capture=True)
arch = arch.strip()
repo_url = urljoin(repo_url, f'{pkg_isc_version}/v{self.revision}/{arch}/')
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
upload_cmd += ' ' + repo_url
for fn in os.listdir(pkgs_dir):
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if file_ext and not fn.endswith(file_ext):
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
continue
fp = os.path.join(pkgs_dir, fn)
cmd = upload_cmd % fp
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
exit_code, txt = execute(cmd, raise_error=False, capture=True, attempts=3)
log.info('code: %s, txt: %s', exit_code, txt)
# debian doc packages is arch "all" in both x86 and aarch so we want to ignore error
# while second is uploaded
if exit_code != 0 and "Repository does not allow updating assets" in txt:
continue
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
t1 = time.time()
dt = int(t1 - t0)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
log.info('Build log file stored to %s', log_file_path)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
log.info("")
log.info(">>>>>> Build time %s:%s", dt // 60, dt % 60)
log.info("")
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# run unit tests if requested
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
total = 0
passed = 0
try:
if 'unittest' in self.features:
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
cmd = 'scp -F %s -r default:/home/vagrant/unit-test-results.json .' % ssh_cfg_path
execute(cmd, cwd=self.vagrant_dir)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
results_file = os.path.join(self.vagrant_dir, 'unit-test-results.json')
if os.path.exists(results_file):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open(results_file, encoding='utf-8') as f:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
txt = f.read()
results = json.loads(txt)
total = results['grand_total']
passed = results['grand_passed']
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
cmd = 'scp -F %s -r default:/home/vagrant/aggregated_tests.xml .' % ssh_cfg_path
execute(cmd, cwd=self.vagrant_dir)
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
except Exception as e:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.exception('ignored issue with parsing unit test results: %s', e)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
return total, passed
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
def destroy(self):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Remove the VM completely."""
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if os.path.exists(self.vagrant_dir):
cmd = 'vagrant destroy --force'
execute(cmd, cwd=self.vagrant_dir, timeout=3 * 60, dry_run=self.dry_run) # timeout: 3 minutes
execute('rm -rf %s' % self.vagrant_dir)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def ssh(self):
[#372,!181] Added many missing comments
2019-01-30 20:44:35 +01:00
"""Open interactive session to the VM."""
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
execute('vagrant ssh', cwd=self.vagrant_dir, timeout=None, dry_run=self.dry_run, interactive=True)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
def dump_ssh_config(self):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Dump ssh config that allows getting into Vagrant system via SSH."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
ssh_cfg_path = os.path.join(self.vagrant_dir, 'ssh.cfg')
execute('vagrant ssh-config > %s' % ssh_cfg_path, cwd=self.vagrant_dir)
return ssh_cfg_path
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
def execute(self, cmd, timeout=None, raise_error=True, log_file_path=None, quiet=False, env=None, capture=False,
[#1433] hammer: fixed passing attempts argument to execute
2020-09-25 10:12:35 +02:00
attempts=1, sleep_time_after_attempt=None):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Execute provided command inside Vagrant system."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if not env:
env = os.environ.copy()
env['LANGUAGE'] = env['LANG'] = env['LC_ALL'] = 'C'
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
return execute('vagrant ssh -c "%s"' % cmd, env=env, cwd=self.vagrant_dir, timeout=timeout,
raise_error=raise_error, dry_run=self.dry_run, log_file_path=log_file_path,
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
quiet=quiet, check_times=self.check_times, capture=capture,
[#1433] hammer: fixed passing attempts argument to execute
2020-09-25 10:12:35 +02:00
attempts=attempts, sleep_time_after_attempt=sleep_time_after_attempt)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
def prepare_system(self):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Prepare Vagrant system for building Kea."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if self.features:
self.features_arg = '--with ' + ' '.join(self.features)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
else:
self.features_arg = ''
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
nofeatures = set(DEFAULT_FEATURES) - self.features
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if nofeatures:
self.nofeatures_arg = '--without ' + ' '.join(nofeatures)
else:
self.nofeatures_arg = ''
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# install python3 on some systems
[#1877] fixed building docs by switching venv from py2 to py3
2021-05-20 08:16:01 +02:00
if self.system == 'centos':
if self.revision == '7':
self.execute("sudo yum remove -y python-devel")
self.execute("sudo yum install -y python36 rpm-build python3-virtualenv", attempts=3)
else:
self.execute("sudo dnf install -y python36 rpm-build python3-virtualenv", attempts=3)
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
elif self.system == 'freebsd':
if self.revision.startswith('13'):
[#3738] Do not update repos on FreeBSD in hammer.py
2025-01-30 14:01:43 +02:00
self.execute("sudo pkg install --no-repo-update --yes python3", attempts=3)
[#934,!531] added support for centos8 rpm
2019-10-14 19:03:46 +07:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# select proper python version for running Hammer inside Vagrant system
[#1877] fixed building docs by switching venv from py2 to py3
2021-05-20 08:16:01 +02:00
if self.system == 'freebsd':
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if self.revision.startswith(('11', '12')):
self.python = 'python3.6'
else:
self.python = 'python3'
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
else:
self.python = 'python3'
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# to get python in RHEL 8 beta it is required first register machine in RHEL account
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
if self.system == 'rhel' and self.revision == '8':
cmd = "sudo subscription-manager repos --list-enabled | grep rhel-8-for-x86_64-baseos-beta-rpms"
exitcode = self.execute(cmd, raise_error=False)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if exitcode != 0:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
env = os.environ.copy()
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open(os.path.expanduser('~/rhel-creds.txt'), encoding='utf-8') as f:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
env['RHEL_USER'] = f.readline().strip()
env['RHEL_PASSWD'] = f.readline().strip()
self.execute('sudo subscription-manager register --user $RHEL_USER --password "$RHEL_PASSWD"', env=env)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
self.execute("sudo subscription-manager refresh")
self.execute("sudo subscription-manager attach --pool 8a85f99a67cdc3e70167e45c85f47429")
self.execute("sudo subscription-manager repos --enable rhel-8-for-x86_64-baseos-beta-rpms")
self.execute("sudo dnf install -y python36")
[#2539] hammer.py: configure RPM distros to look for libraries in /usr/local/lib64
2022-09-05 22:30:32 +03:00
# RPM-based distributions install libraries in /usr/local/lib64, but they
# tend to not look there at runtime without explicit mention in ld.so.conf.d.
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if self.system in ['centos', 'fedora', 'rhel', 'rocky']:
[#2539] hammer.py: configure RPM distros to look for libraries in /usr/local/lib64
2022-09-05 22:30:32 +03:00
self.execute('sudo echo /usr/local/lib64 > /etc/ld.so.conf.d/kea.conf')
# ldconfig only in case the change above was not there before system startup
self.execute('sudo ldconfig')
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# upload Hammer to Vagrant system
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hmr_py_path = os.path.join(self.repo_dir, 'hammer.py')
hammer: hardened vagrant upload, tuned timeouts
2019-01-24 15:32:03 +01:00
self.upload(hmr_py_path)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
log_file_path = os.path.join(self.vagrant_dir, 'prepare.log')
log.info('Prepare log file stored to %s', log_file_path)
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
t0 = time.time()
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# run prepare-system inside Vagrant system
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
cmd = "{python} hammer.py prepare-system -p local {features} {nofeatures} {check_times} {ccache}"
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
cmd = cmd.format(python=self.python,
features=self.features_arg,
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
nofeatures=self.nofeatures_arg,
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
check_times='-i' if self.check_times else '',
ccache='--ccache-dir /ccache' if self.ccache_enabled else '')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
self.execute(cmd, timeout=40 * 60, log_file_path=log_file_path, quiet=self.quiet)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
t1 = time.time()
dt = int(t1 - t0)
log.info('')
log.info(">>> Preparing %s, %s, %s completed in %s:%s", self.provider, self.system, self.revision,
dt // 60, dt % 60)
log.info('')
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
def prepare_for_boxing(self):
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if self.system in ['debian', 'ubuntu', 'fedora', 'centos', 'rhel', 'rocky']:
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# setup a script that on first boot will set machine-id
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = 'sh -c \'cat <<EOF | sudo tee /usr/lib/systemd/system/systemd-firstboot.service\n'
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
cmd += '[Unit]\n'
cmd += 'Description=Generate New Machine ID\n'
cmd += 'Documentation=man:systemd-firstboot(1)\n'
cmd += 'DefaultDependencies=no\n'
cmd += 'Conflicts=shutdown.target\n'
cmd += 'After=systemd-remount-fs.service\n'
cmd += 'Before=systemd-sysusers.service sysinit.target shutdown.target\n'
cmd += 'ConditionPathIsReadWrite=/etc\n'
cmd += 'ConditionFirstBoot=yes\n'
cmd += '[Service]\n'
cmd += 'Type=oneshot\n'
cmd += 'RemainAfterExit=yes\n'
cmd += 'ExecStart=/usr/bin/systemd-firstboot --setup-machine-id\n'
cmd += '[Install]\n'
cmd += 'WantedBy=sysinit.target\n'
cmd += "EOF\n\'"
self.execute(cmd)
self.execute('sudo systemctl enable systemd-firstboot.service')
elif self.system == 'freebsd':
self.execute('sudo pkg clean -a -y')
self.execute('sudo rm -rf /usr/lib/debug')
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def _install_gtest_sources():
[#372,!181] Added many missing comments
2019-01-30 20:44:35 +01:00
"""Install gtest sources."""
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# download gtest sources only if it is not present as native package
[#3350] Updated gtest version to 1.14
2024-04-24 11:41:18 +00:00
gtest_version = '1.14.0'
[#2539] mostly cosmetical changes to hammer.py
2022-09-05 19:32:22 +03:00
gtest_path = f'/usr/src/googletest-release-{gtest_version}/googletest'
if os.path.exists(gtest_path):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('gtest is already installed in %s.', gtest_path)
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
return
[#2539] mostly cosmetical changes to hammer.py
2022-09-05 19:32:22 +03:00
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('mkdir -p ~/.hammer-tmp')
cmd = 'wget --no-verbose -O ~/.hammer-tmp/gtest.tar.gz '
[#3350] Updated gtest version to 1.14
2024-04-24 11:41:18 +00:00
cmd += f'https://github.com/google/googletest/archive/refs/tags/v{gtest_version}.tar.gz'
[#2539] mostly cosmetical changes to hammer.py
2022-09-05 19:32:22 +03:00
execute(cmd)
execute('sudo mkdir -p /usr/src')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('sudo tar -C /usr/src -zxf ~/.hammer-tmp/gtest.tar.gz')
[#3350] Updated gtest version to 1.14
2024-04-24 11:41:18 +00:00
execute(f'sudo ln -sf /usr/src/googletest-{gtest_version} /usr/src/googletest')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
def _install_libyang_from_sources(ignore_errors=False):
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
"""Install libyang from sources."""
for prefix in ['/usr', '/usr/local']:
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
libyang_so_candidates = [f'{prefix}/lib/libyang.so', f'{prefix}/lib64/libyang.so']
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
libyang_header = f'{prefix}/include/libyang/version.h'
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if (any(os.path.exists(i) for i in libyang_so_candidates) and os.path.exists(libyang_header) and
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute(f"grep -F '#define LY_VERSION_MAJOR 2' '{libyang_header}'", raise_error=False) == 0):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('libyang is already installed at %s.', libyang_header)
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
return
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
version = 'v2.1.4'
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
execute('mkdir -p ~/.hammer-tmp')
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
try:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('git clone https://github.com/CESNET/libyang.git ~/.hammer-tmp/libyang')
execute(f'git checkout {version}', cwd='~/.hammer-tmp/libyang')
execute('mkdir ~/.hammer-tmp/libyang/build')
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute('cmake -DBUILD_TESTING=OFF -DCMAKE_C_FLAGS="-Wno-incompatible-pointer-types" ..',
cwd='~/.hammer-tmp/libyang/build')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('make -j $(nproc || gnproc || echo 1)', cwd='~/.hammer-tmp/libyang/build')
execute('sudo make install', cwd='~/.hammer-tmp/libyang/build')
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
system, _ = get_system_revision()
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if system != 'alpine':
execute('sudo ldconfig')
except Exception as e:
log.exception(str(e))
if not ignore_errors:
raise e
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
finally:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
def _install_sysrepo_from_sources(ignore_errors=False):
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
"""Install sysrepo from sources."""
for prefix in ['/usr', '/usr/local']:
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
sysrepo_so_candidates = [f'{prefix}/lib/libsysrepo.so', f'{prefix}/lib64/libsysrepo.so']
[#2601] fix typos in NETCONF code
2022-11-08 12:12:53 +02:00
sysrepo_header = f'{prefix}/include/sysrepo/version.h'
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if (any(os.path.exists(i) for i in sysrepo_so_candidates) and os.path.exists(sysrepo_header) and
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute(f"grep -F '#define SR_VERSION_MAJOR 7' '{sysrepo_header}'", raise_error=False) == 0):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('sysrepo is already installed at %s.', sysrepo_header)
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
return
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
version = 'v2.2.12'
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[#2049] hammer.py: create YANG repository
2021-09-18 10:10:01 +03:00
# Create repository for YANG modules and change ownership to current user.
execute('sudo mkdir -p /etc/sysrepo')
[qa#276] fix hammer's netconf and postgres actions on Fedora 33 and FreeBSDs
2021-11-10 21:04:19 +02:00
execute('sudo chown -R "${USER}:$(id -gn)" /etc/sysrepo')
[#2049] hammer.py: create YANG repository
2021-09-18 10:10:01 +03:00
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
execute('mkdir -p ~/.hammer-tmp')
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
try:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('git clone https://github.com/sysrepo/sysrepo.git ~/.hammer-tmp/sysrepo')
execute(f'git checkout {version}', cwd='~/.hammer-tmp/sysrepo')
execute('mkdir ~/.hammer-tmp/sysrepo/build')
[#3393] ignore error in libyang tests ... and add bison and flex to ubuntu for completion
2024-05-17 08:47:55 +03:00
execute('cmake -DBUILD_TESTING=OFF -DREPO_PATH=/etc/sysrepo ..', cwd='~/.hammer-tmp/sysrepo/build')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('make -j $(nproc || gnproc || echo 1)', cwd='~/.hammer-tmp/sysrepo/build')
execute('sudo make install', cwd='~/.hammer-tmp/sysrepo/build')
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
system, _ = get_system_revision()
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if system != 'alpine':
execute('sudo ldconfig')
except Exception as e:
log.exception(str(e))
if not ignore_errors:
raise e
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
finally:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
def _install_libyang_cpp_from_sources(ignore_errors=False):
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
"""Install libyang-cpp from sources."""
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
for prefix_lib in ['/usr/lib', '/usr/lib64', '/usr/local/lib', '/usr/local/lib64']:
libyang_cpp_so = f'{prefix_lib}/libyang-cpp.so'
libyang_cpp_pc = f'{prefix_lib}/pkgconfig/libyang-cpp.pc'
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if (os.path.exists(libyang_cpp_so) and os.path.exists(libyang_cpp_pc) and
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute(f"grep -F 'Version: 1.1.0' '{libyang_cpp_pc}'", raise_error=False) == 0):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('libyang-cpp is already installed at %s.', libyang_cpp_so)
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
return
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
version = 'ae7d649ea75da081725c119dd553b2ef3121a6f8'
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
execute('mkdir -p ~/.hammer-tmp')
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
try:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('git clone https://github.com/CESNET/libyang-cpp.git ~/.hammer-tmp/libyang-cpp')
execute(f'git checkout {version}', cwd='~/.hammer-tmp/libyang-cpp')
[#3393] add fedora 40 to hammer
2024-05-13 14:41:05 +00:00
# New cpp compiler is more picky about missing headers. (ex. Fedora 40)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
execute("""git apply <<EOF
diff --git a/src/Context.cpp b/src/Context.cpp
index b2fe887..add11cc 100644
--- a/src/Context.cpp
+++ b/src/Context.cpp
@@ -13,2 +13,3 @@
#include <libyang/libyang.h>
+#include <algorithm>
#include <span>
EOF
""", cwd='~/.hammer-tmp/libyang-cpp')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('mkdir ~/.hammer-tmp/libyang-cpp/build')
execute('cmake -DBUILD_TESTING=OFF .. ', cwd='~/.hammer-tmp/libyang-cpp/build')
execute('make -j $(nproc || gnproc || echo 1)', cwd='~/.hammer-tmp/libyang-cpp/build')
execute('sudo make install', cwd='~/.hammer-tmp/libyang-cpp/build')
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
system, _ = get_system_revision()
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if system != 'alpine':
execute('sudo ldconfig')
except Exception as e:
log.exception(str(e))
if not ignore_errors:
raise e
finally:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
def _install_sysrepo_cpp_from_sources(ignore_errors=False):
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
"""Install sysrepo-cpp from sources."""
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
for prefix_lib in ['/usr/lib', '/usr/lib64', '/usr/local/lib', '/usr/local/lib64']:
sysrepo_cpp_so = f'{prefix_lib}/libsysrepo-cpp.so'
sysrepo_cpp_pc = f'{prefix_lib}/pkgconfig/sysrepo-cpp.pc'
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if (os.path.exists(sysrepo_cpp_so) and os.path.exists(sysrepo_cpp_pc) and
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute(f"grep -F 'Version: 1.1.0' '{sysrepo_cpp_pc}'", raise_error=False) == 0):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('sysrepo-cpp is already installed at %s.', sysrepo_cpp_so)
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
return
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
version = '02634174ffc60568301c3d9b9b7cf710cff6a586'
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
execute('mkdir -p ~/.hammer-tmp')
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
try:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('git clone https://github.com/sysrepo/sysrepo-cpp.git ~/.hammer-tmp/sysrepo-cpp')
execute(f'git checkout {version}', cwd='~/.hammer-tmp/sysrepo-cpp')
execute('mkdir ~/.hammer-tmp/sysrepo-cpp/build')
execute('cmake -DBUILD_TESTING=OFF .. ', cwd='~/.hammer-tmp/sysrepo-cpp/build')
execute('make -j $(nproc || gnproc || echo 1)', cwd='~/.hammer-tmp/sysrepo-cpp/build')
execute('sudo make install', cwd='~/.hammer-tmp/sysrepo-cpp/build')
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
system, _ = get_system_revision()
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if system != 'alpine':
execute('sudo ldconfig')
except Exception as e:
log.exception(str(e))
if not ignore_errors:
raise e
finally:
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('rm -rf ~/.hammer-tmp')
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
def _install_netconf_libraries_from_sources(ignore_errors=False):
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
_install_libyang_from_sources(ignore_errors)
_install_sysrepo_from_sources(ignore_errors)
_install_libyang_cpp_from_sources(ignore_errors)
_install_sysrepo_cpp_from_sources(ignore_errors)
[#1811] hammer.py: set timezone in postgres
2021-04-19 15:41:40 +03:00
def _get_local_timezone():
[#1811] hammer.py: set timezone in postgres
2021-04-19 12:42:30 +03:00
_, output = execute('''
# timedatectl
if command -v timedatectl > /dev/null 2>&1; then
timedatectl | grep 'Time zone' | cut -d ':' -f 2 | cut -d '(' -f 1 | xargs
exit ${?}
fi
[#1811] hammer.py: set timezone in postgres
2021-04-19 15:41:40 +03:00
# brute force: for Alpine, FreeBSD
[#1953] hammer.py: Continent/Country/Zone TZ
2021-06-29 13:21:48 +03:00
for i in $(find /usr/share/zoneinfo | cut -d '/' -f 5- | grep -E '^[A-Z][a-z]+/[A-Z][a-z]+.*$' | sort -ruV); do
[#1811] hammer.py: set timezone in postgres
2021-04-19 12:42:30 +03:00
unset TZ
if test "$(date)" = "$(TZ="${i}" date)"; then
printf '%s\n' "${i}"
break
fi
done
''', capture=True, quiet=True)
[#1811] hammer.py: set timezone in postgres
2021-04-19 15:41:40 +03:00
return output.rstrip()
[#1811] hammer.py: set timezone in postgres
2021-04-19 12:42:30 +03:00
hammer: added configuring for forge
2019-01-24 17:54:20 +01:00
def _configure_mysql(system, revision, features):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Configure MySQL database."""
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
# Find MySQL's configuration directory which differs on various systems.
conf_d = None
for i in ['/etc/mysql/conf.d', '/etc/my.cnf.d']:
if os.path.isdir(i):
conf_d = i
break
if conf_d is None:
# No configuration directory found. This happens on some systems like
# Alpine. Consider /etc/my.cnf.d as default.
conf_d = '/etc/my.cnf.d'
execute('sudo mkdir -p {}'.format(conf_d))
# Some systems like Alpine only listen on the unix socket and have to have
# the bind-address configured manually.
return_code = execute("sudo grep -Er '^bind-address' {}".format(conf_d), raise_error=False)
if return_code != 0:
execute("printf '[mysqld]\nbind-address = 127.0.0.1\n' > ./bind-address.cnf")
bind_address_cnf = os.path.join(conf_d, 'bind-address.cnf')
execute('sudo cp ./bind-address.cnf {}'.format(bind_address_cnf))
execute('sudo chown mysql:mysql {}'.format(bind_address_cnf))
execute('sudo rm -f ./bind-address.cnf')
# If requested, configure TLS.
cert_dir = '/etc/mysql/ssl'
kea_cnf = os.path.join(conf_d, 'kea.cnf')
# But start fresh first. Not enabling TLS in hammer leaves TLS support removed.
execute('sudo rm -rf {} {}'.format(cert_dir, kea_cnf))
if 'tls' in features:
if not os.path.isdir(cert_dir):
execute('sudo mkdir -p {}'.format(cert_dir))
[#2969] regenerate key to avoid SSL errors in DB Prevents this error that happens on Debian 10: - SSL error: Unable to get private key from '/etc/mysql/ssl/kea-server.key'
2023-11-03 15:03:29 +02:00
# Some systems, usually old ones, might require a cerain PKCS format
# of the key. Try to regenerate it here, but don't stop if it fails.
# If the key is wrong, it will fail later anyway.
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
exit_code = execute('openssl rsa -in src/lib/asiolink/testutils/ca/kea-server.key '
'-out src/lib/asiolink/testutils/ca/kea-server.key', raise_error=False)
[#2969] regenerate key to avoid SSL errors in DB Prevents this error that happens on Debian 10: - SSL error: Unable to get private key from '/etc/mysql/ssl/kea-server.key'
2023-11-03 15:03:29 +02:00
if exit_code != 0:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.warning('openssl command failed with exit code %d, but continuing...', exit_code)
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
for file in [
'./src/lib/asiolink/testutils/ca/kea-ca.crt',
'./src/lib/asiolink/testutils/ca/kea-client.crt',
'./src/lib/asiolink/testutils/ca/kea-client.key',
'./src/lib/asiolink/testutils/ca/kea-server.crt',
'./src/lib/asiolink/testutils/ca/kea-server.key',
]:
if not os.path.exists(file):
print('ERROR: File {} is needed to prepare TLS.'.format(file), file=sys.stderr)
sys.exit(1)
basename = os.path.basename(file)
execute('sudo cp {} {}'.format(file, os.path.join(cert_dir, basename)))
with open('kea.cnf', 'w', encoding='utf-8') as f:
f.write('''\
[mysqld]
ssl_ca = {cert_dir}/kea-ca.crt
ssl_cert = {cert_dir}/kea-server.crt
ssl_key = {cert_dir}/kea-server.key
[client-mariadb]
ssl_ca = {cert_dir}/kea-ca.crt
ssl_cert = {cert_dir}/kea-client.crt
ssl_key = {cert_dir}/kea-client.key
'''.format(cert_dir=cert_dir))
execute('sudo mv ./kea.cnf {}'.format(kea_cnf))
# For all added files and directories, change owner to mysql.
execute('sudo chown -R mysql:mysql {} {}'.format(cert_dir, kea_cnf))
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if system in ['debian', 'fedora', 'centos', 'rhel', 'rocky']:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
execute('sudo systemctl enable mariadb.service')
[#2969] hammer.py: log error on systemctl restart
2023-10-30 16:46:47 +02:00
exit_code = execute('sudo systemctl restart mariadb.service', raise_error=False)
if exit_code != 0:
log.error('Command "sudo systemctl restart mariadb.service" failed. Here is the journal:')
execute('sudo journalctl -xu mariadb.service', raise_error=False)
log.error('And here are the logs:')
execute('cat /var/log/mysql/error.log', raise_error=False)
sys.exit(exit_code)
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
[#2064] hammer.py: Debian uses mariadb.service
2021-09-06 22:30:10 +03:00
elif system == 'ubuntu':
[#2064] hammer.py: restart mysql on deb OSs
2021-09-03 14:59:12 +03:00
execute('sudo systemctl enable mysql.service')
[#2969] hammer.py: log error on systemctl restart
2023-10-30 16:46:47 +02:00
exit_code = execute('sudo systemctl restart mysql.service', raise_error=False)
if exit_code != 0:
log.error('Command "sudo systemctl restart mysql.service" failed. Here is the journal:')
execute('sudo journalctl -xu mysql.service', raise_error=False)
log.error('And here are the logs:')
execute('cat /var/log/mysql/error.log', raise_error=False)
sys.exit(exit_code)
[#2064] hammer.py: restart mysql on deb OSs
2021-09-03 14:59:12 +03:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
elif system == 'alpine':
[#2503] Remove skip-networking from Alpine mariadb configuration in hammer
2022-07-22 01:14:23 -07:00
execute('sudo sed -i "/^skip-networking$/d" /etc/my.cnf.d/mariadb-server.cnf')
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute('sudo rc-update add mariadb')
[kea-packaging#8] hammer: stop mariadb before setup Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-13 11:23:15 +03:00
execute('sudo rc-service mariadb stop')
[#3559] hammer.py: pkill mariadb_safe before restarting Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-27 17:36:08 +03:00
wait_for_process_to_start('start-stop-daemon') # mysqld_safe
[#3547] hammer.py: wait for start-stop-daemon/mysqld_safe to exit Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-22 10:23:46 +03:00
wait_for_process_to_exit('start-stop-daemon') # mysqld_safe
[#3559] hammer.py: pkill mariadb_safe before restarting Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-27 17:36:08 +03:00
execute('sudo pkill -f mysqld_safe', raise_error=False) # If no graceful shutdown, force kill.
[kea-packaging#8] hammer: stop mariadb before setup Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-13 11:23:15 +03:00
execute('sudo rc-service mariadb setup')
[#3559] hammer.py: pkill mariadb_safe before restarting Attempt to fix `start-stop-daemon: /usr/bin/mysqld_safe is already running`
2024-08-27 17:36:08 +03:00
execute('sudo rc-service mariadb restart', raise_error=False)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
cmd = "echo 'DROP DATABASE IF EXISTS keatest;' | sudo mysql -u root"
execute(cmd)
cmd = "echo 'DROP USER 'keatest'@'localhost';' | sudo mysql -u root"
execute(cmd, raise_error=False)
cmd = "echo 'DROP USER 'keatest_readonly'@'localhost';' | sudo mysql -u root"
execute(cmd, raise_error=False)
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
cmd = "echo 'DROP USER 'keatest_secure'@'localhost';' | sudo mysql -u root"
execute(cmd, raise_error=False)
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = "sh -c \"cat <<EOF | sudo mysql -u root\n"
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
cmd += "CREATE DATABASE keatest;\n"
cmd += "CREATE USER 'keatest'@'localhost' IDENTIFIED BY 'keatest';\n"
cmd += "CREATE USER 'keatest_readonly'@'localhost' IDENTIFIED BY 'keatest';\n"
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
if 'tls' in features:
cmd += "CREATE USER 'keatest_secure'@'localhost' IDENTIFIED BY 'keatest';\n"
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
cmd += "GRANT ALL ON keatest.* TO 'keatest'@'localhost';\n"
cmd += "GRANT SELECT ON keatest.* TO 'keatest_readonly'@'localhost';\n"
cmd += "EOF\n\""
execute(cmd)
[#2353] hammer.py: try both ALTER and GRANT for REQUIRE X509
2022-04-07 11:14:12 +03:00
if 'tls' in features:
# ALTER USER is the best place to put the REQUIRE but, if it is not
# supported, then downgrade to GRANT.
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
exit_code = execute('''sudo mysql -u root -e "ALTER USER 'keatest_secure'@'localhost' REQUIRE X509;"''',
raise_error=False)
[#2353] hammer.py: try both ALTER and GRANT for REQUIRE X509
2022-04-07 11:14:12 +03:00
if exit_code == 0:
# If ALTER succeeds, then we still have to GRANT without REQUIRE.
execute('''sudo mysql -u root -e "GRANT ALL ON keatest.* TO 'keatest_secure'@'localhost';"''')
else:
execute('''sudo mysql -u root -e "GRANT ALL ON keatest.* TO 'keatest_secure'@'localhost' REQUIRE X509;"''')
execute('sudo mysql -u root -e "SET @@global.log_bin_trust_function_creators = 1;"')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
hammer: added configuring for forge
2019-01-24 17:54:20 +01:00
if 'forge' in features:
cmd = "echo 'DROP DATABASE IF EXISTS keadb;' | sudo mysql -u root"
execute(cmd)
cmd = "echo 'DROP USER 'keauser'@'localhost';' | sudo mysql -u root"
execute(cmd, raise_error=False)
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = "sh -c \"cat <<EOF | sudo mysql -u root\n"
hammer: added configuring for forge
2019-01-24 17:54:20 +01:00
cmd += "CREATE DATABASE keadb;\n"
cmd += "CREATE USER 'keauser'@'localhost' IDENTIFIED BY 'keapass';\n"
cmd += "GRANT ALL ON keadb.* TO 'keauser'@'localhost';\n"
cmd += "EOF\n\""
execute(cmd)
hammer: various improvements - added sql script for mysql used on debian 9 to fix MariaDB issue - added enabling kea-shell feature - removed hack for detecting boost on FreeBSD as it is fixed in Kea now
2019-01-23 11:26:33 +01:00
if system == 'debian' and revision == '9':
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('FIX FOR ISSUE kea#389: %s %s', system, revision)
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = "sh -c \"cat <<EOF | sudo mysql -u root\n"
hammer: various improvements - added sql script for mysql used on debian 9 to fix MariaDB issue - added enabling kea-shell feature - removed hack for detecting boost on FreeBSD as it is fixed in Kea now
2019-01-23 11:26:33 +01:00
cmd += "use keatest;\n"
cmd += "set global innodb_large_prefix=on;\n"
cmd += "set global innodb_file_format=Barracuda;\n"
cmd += "set global innodb_file_per_table=true;\n"
cmd += "set global innodb_default_row_format=dynamic;\n"
cmd += "EOF\n\""
execute(cmd)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
def _enable_postgresql(system):
[#1814] hammer: cwd=tmp to solve permission denied
2021-04-20 13:06:24 +03:00
if system == 'alpine':
execute('sudo rc-update add postgresql')
Fix the PostgreSQL initial setup issues on FreeBSD
2021-05-04 22:05:27 +02:00
elif system == 'freebsd':
execute('sudo sysrc postgresql_enable="yes"')
[#1814] hammer: cwd=tmp to solve permission denied
2021-04-20 13:06:24 +03:00
else:
[#2539] hammer.py: switch from PostgreSQL 14 to the default in RHEL 9 There is a lot of code that deals with previously installed versions of PostgreSQL. This is good since it deals with a lot of use cases and allows us to not regenerate images, but it also adds complexity.
2022-09-05 21:59:12 +03:00
# Disable all PostgreSQL services first to avoid conflicts.
# raise_error=False for when there are no matches
_, output = execute("systemctl list-unit-files | grep postgres | grep -Fv '@.service' | cut -d ' ' -f 1",
capture=True, raise_error=False)
for service in output.split():
execute(f'sudo systemctl disable {service}')
[#1814] hammer: cwd=tmp to solve permission denied
2021-04-20 13:06:24 +03:00
execute('sudo systemctl enable postgresql.service')
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
def _restart_postgresql(system):
[#1814] hammer: get pg_hba location automatically
2021-04-20 09:04:42 +03:00
if system == 'freebsd':
# redirecting output from start script to /dev/null otherwise the postgresql rc.d script will hang
# calling restart instead of start allow hammer.py to pass even if postgresql is already installed
execute('sudo service postgresql restart > /dev/null')
elif system == 'alpine':
execute('sudo /etc/init.d/postgresql restart')
else:
[#2539] hammer.py: switch from PostgreSQL 14 to the default in RHEL 9 There is a lot of code that deals with previously installed versions of PostgreSQL. This is good since it deals with a lot of use cases and allows us to not regenerate images, but it also adds complexity.
2022-09-05 21:59:12 +03:00
# Stop all PostgreSQL services first to avoid conflicts.
# raise_error=False for when there are no matches
_, output = execute("systemctl list-unit-files | grep postgres | grep -Fv '@.service' | cut -d ' ' -f 1",
capture=True, raise_error=False)
for service in output.split():
execute(f'sudo systemctl stop {service}')
exit_code = execute('sudo systemctl restart postgresql.service', raise_error=False)
if exit_code != 0:
log.error('Command "sudo systemctl restart postgresql.service" failed. Here is the journal:')
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
execute('sudo journalctl -xu postgresql.service', raise_error=False)
[#2969] hammer.py: log error on systemctl restart
2023-10-30 16:46:47 +02:00
log.error('And here are the logs:')
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
_, output = execute("sudo -u postgres psql -A -t -c 'SELECT pg_current_logfile()'",
capture=True, quiet=True)
[#2539] hammer.py: switch from PostgreSQL 14 to the default in RHEL 9 There is a lot of code that deals with previously installed versions of PostgreSQL. This is good since it deals with a lot of use cases and allows us to not regenerate images, but it also adds complexity.
2022-09-05 21:59:12 +03:00
logfile = os.path.basename(output.strip())
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
execute(fr'sudo find /var -type f -name "{logfile}" -exec cat {{}} \;', raise_error=False)
[#2539] hammer.py: switch from PostgreSQL 14 to the default in RHEL 9 There is a lot of code that deals with previously installed versions of PostgreSQL. This is good since it deals with a lot of use cases and allows us to not regenerate images, but it also adds complexity.
2022-09-05 21:59:12 +03:00
sys.exit(exit_code)
[#1814] hammer: get pg_hba location automatically
2021-04-20 09:04:42 +03:00
[#2064] hammer: don't modify postgres entry in hba
2021-09-03 15:00:03 +03:00
# Change authentication type for given connection type. Usual inputs for
# connection type are 'host' or 'local'. Only affects entries with database
# and user both set to 'all'. This is to not affect authentication of
# `postgres` user which should have a separate entry.
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
def _change_postgresql_auth_method(connection_type, auth_method, hba_file):
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
execute(fr"sudo sed -i.bak 's/^{connection_type}\(.*\)all\(.*\)all\(.*\) [a-z0-9]*$"
fr"/{connection_type}\1all\2all\3 {auth_method}/g' '{hba_file}'",
cwd='/tmp')
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
def _configure_pgsql(system, features):
Added support for PostgreSQL DB on FreeBSD systems
2019-08-10 21:47:38 +02:00
""" Configure PostgreSQL DB """
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
# Some execute() calls set cwd='/tmp' when switching user to postgres to
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
# avoid the error:
# could not change as postgres user directory to "/home/jenkins": Permission denied
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if system in ['fedora', 'centos', 'rhel', 'rocky']:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
# https://fedoraproject.org/wiki/PostgreSQL
hammer: fixed missing arg
2019-01-11 16:32:55 +01:00
exitcode = execute('sudo ls /var/lib/pgsql/data/postgresql.conf', raise_error=False)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if exitcode != 0:
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if system == 'centos':
execute('sudo postgresql-setup initdb')
else:
execute('sudo postgresql-setup --initdb --unit postgresql')
Added support for PostgreSQL DB on FreeBSD systems
2019-08-10 21:47:38 +02:00
elif system == 'freebsd':
[#3470] make hammer.py work better with freebsd 14+ - Remove unnecessary `SET PASSWORD` for mysql - Remove redundant `sysrc postgresql_enable=yes`. It is done later when enabling the service. - Determine `/var/db/postgres/data*` using glob. - Add `interactive=true` to onestart command.
2024-06-27 15:43:30 +03:00
# If data directory is not created, then initdb.
var_db_postgres_data = glob.glob('/var/db/postgres/data*')
if len(var_db_postgres_data) == 0:
execute('sudo service postgresql oneinitdb')
[qa#276] fix hammer's netconf and postgres actions on Fedora 33 and FreeBSDs
2021-11-10 21:04:19 +02:00
[#3470] make hammer.py work better with freebsd 14+ - Remove unnecessary `SET PASSWORD` for mysql - Remove redundant `sysrc postgresql_enable=yes`. It is done later when enabling the service. - Determine `/var/db/postgres/data*` using glob. - Add `interactive=true` to onestart command.
2024-06-27 15:43:30 +03:00
# Get data directory again. It will be needed later.
var_db_postgres_data = glob.glob('/var/db/postgres/data*')
if len(var_db_postgres_data) == 0:
raise UnexpectedError('Could not find /var/db/postgres/data*')
var_db_postgres_data = var_db_postgres_data[-1]
[qa#276] fix hammer's netconf and postgres actions on Fedora 33 and FreeBSDs
2021-11-10 21:04:19 +02:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
# if the file '/var/db/postgres/data*/postmaster.opts' does not exist the 'restart' of postgresql will fail
# with error:
[qa#276] fix hammer's netconf and postgres actions on Fedora 33 and FreeBSDs
2021-11-10 21:04:19 +02:00
# pg_ctl: could not read file "/var/db/postgres/data*/postmaster.opts"
Fix the PostgreSQL initial setup issues on FreeBSD
2021-05-04 22:05:27 +02:00
# the initial start of the postgresql will create the 'postmaster.opts' file
[#3470] make hammer.py work better with freebsd 14+ - Remove unnecessary `SET PASSWORD` for mysql - Remove redundant `sysrc postgresql_enable=yes`. It is done later when enabling the service. - Determine `/var/db/postgres/data*` using glob. - Add `interactive=true` to onestart command.
2024-06-27 15:43:30 +03:00
# There might be a bug that makes execute freeze when the subprocess exits before reaching communicate().
# In reality, this should never happen. I'm suspecting a bug in python. interactive=True prevents the freeze.
execute(f'if sudo test ! -f {var_db_postgres_data}/postmaster.opts; then sudo service postgresql onestart; fi',
interactive=True)
Added support for PostgreSQL DB on FreeBSD systems
2019-08-10 21:47:38 +02:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
_enable_postgresql(system)
_restart_postgresql(system)
[#1814] hammer: get pg_hba location automatically
2021-04-20 09:04:42 +03:00
[#2064] hammer: don't modify postgres entry in hba
2021-09-03 15:00:03 +03:00
# Change auth-method to 'md5' on all connections.
cmd = "sudo -u postgres psql -t -c 'SHOW hba_file' | xargs"
_, output = execute(cmd, capture=True, cwd='/tmp')
hba_file = output.rstrip()
_change_postgresql_auth_method('host', 'md5', hba_file)
_change_postgresql_auth_method('local', 'md5', hba_file)
[#2064] hammer.py: ensure postgres entry in hba
2021-09-06 20:28:26 +03:00
# Make sure hba file has a postgres superuser entry. It needs to be placed
# before any other local auth method for higher priority. Let's simulate
# that by putting it just after the auth header.
if 0 != execute("sudo cat {} | grep -E '^local.*all.*postgres'".format(hba_file), raise_error=False):
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
auth_header = '# TYPE DATABASE USER ADDRESS METHOD'
postgres_auth_line = 'local all postgres ident'
[qa#276] fix hammer's netconf and postgres actions on Fedora 33 and FreeBSDs
2021-11-10 21:04:19 +02:00
# The "\\" followed by newline is for BSD support.
execute("""sudo sed -i.bak '/{}/a\\
{}
' '{}'""".format(auth_header, postgres_auth_line, hba_file))
[#2064] hammer.py: ensure postgres entry in hba
2021-09-06 20:28:26 +03:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
_restart_postgresql(system)
[#2064] hammer: don't modify postgres entry in hba
2021-09-03 15:00:03 +03:00
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = """sh -c \"cat <<EOF | sudo -u postgres psql postgres
[#1811] hammer.py: postgres read only setup
2021-04-16 14:03:52 +03:00
DROP DATABASE IF EXISTS keatest;
DROP USER IF EXISTS keatest;
DROP USER IF EXISTS keatest_readonly;
CREATE USER keatest WITH PASSWORD 'keatest';
CREATE USER keatest_readonly WITH PASSWORD 'keatest';
CREATE DATABASE keatest;
GRANT ALL PRIVILEGES ON DATABASE keatest TO keatest;
[#1811] hammer.py: set timezone in postgres
2021-04-19 15:41:40 +03:00
ALTER DATABASE keatest SET TIMEZONE='{}';\n""".format(_get_local_timezone())
[#1811] hammer.py: postgres read only setup
2021-04-16 14:03:52 +03:00
cmd += 'EOF\n"'
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
execute(cmd, cwd='/tmp')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#3418] remove os check before Postgres ddb grant
2024-05-20 14:15:40 +00:00
# This is needed for postgres >= 15
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = """sh -c \"cat <<EOF | sudo -u postgres psql -U postgres -d keatest
[#2773] Update Hammer queries
2024-05-22 12:31:30 +02:00
GRANT ALL PRIVILEGES ON SCHEMA public TO keatest;\n"""
[#3418] remove os check before Postgres ddb grant
2024-05-20 14:15:40 +00:00
cmd += 'EOF\n"'
execute(cmd, cwd='/tmp')
[#2950] fix pgsql 15 setup for unit tests
2023-06-28 11:19:15 +02:00
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = """sh -c \"cat <<EOF | sudo -u postgres psql -U keatest keatest
[#1814] hammer: cwd=tmp to solve permission denied
2021-04-20 12:53:54 +03:00
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO keatest_readonly;\n"""
cmd += 'EOF\n"'
[#1814] hammer: cwd=tmp to solve permission denied
2021-04-20 13:06:24 +03:00
env = os.environ.copy()
env['PGPASSWORD'] = 'keatest'
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
execute(cmd, cwd='/tmp', env=env)
[#1811] hammer.py: postgres read only setup
2021-04-16 14:03:52 +03:00
hammer: added configuring for forge
2019-01-24 17:54:20 +01:00
if 'forge' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
cmd = "sh -c \"cat <<EOF | sudo -u postgres psql postgres\n"
hammer: added configuring for forge
2019-01-24 17:54:20 +01:00
cmd += "DROP DATABASE IF EXISTS keadb;\n"
cmd += "DROP USER IF EXISTS keauser;\n"
cmd += "CREATE USER keauser WITH PASSWORD 'keapass';\n"
cmd += "CREATE DATABASE keadb;\n"
cmd += "GRANT ALL PRIVILEGES ON DATABASE keauser TO keadb;\n"
cmd += "EOF\n\""
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
execute(cmd, cwd='/tmp')
[#3630] forge db init fix
2024-10-23 11:48:30 +00:00
# This is needed for postgres >= 15
cmd = """sh -c \"cat <<EOF | sudo -u postgres psql -U postgres -d keadb
GRANT ALL PRIVILEGES ON SCHEMA public TO keauser;\n"""
cmd += 'EOF\n"'
execute(cmd, cwd='/tmp')
[#2064] hammer: change pg auth method to md5
2021-09-01 13:39:57 +03:00
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
log.info('postgresql just configured')
hammer: added configuring for forge
2019-01-24 17:54:20 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
hammer: fixed building on debian buster
2019-07-22 10:52:40 +02:00
def _apt_update(system, revision, env=None, check_times=False, attempts=1, sleep_time_after_attempt=None,
capture=False):
cmd = 'sudo apt update'
if system == 'debian' and int(revision) >= 10:
cmd += ' --allow-releaseinfo-change'
return execute(cmd, env=env, check_times=check_times, attempts=attempts,
sleep_time_after_attempt=sleep_time_after_attempt, capture=capture)
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
def _get_package_version(package: str):
"""
Returns the version available in the package manager's repository for the requested package.
:param package: the name of the package whose version is retrieved
"""
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
system, _ = get_system_revision()
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if system == 'alpine':
cmd = "apk search --exact {0} | sed 's/{0}-//g'"
elif system in ['debian', 'ubuntu']:
cmd = "apt-cache show {} | grep -F 'Version:' | cut -d ' ' -f 2"
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
elif system in ['centos', 'fedora', 'rhel', 'rocky']:
cmd = "dnf list {} -y | tr -s ' ' | cut -d ' ' -f 2 | tail -n 1"
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
elif system == 'freebsd':
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
cmd = r"pkg search {0} | grep -Eo '^{0}-[0-9_,\.]+' | sed 's/{0}-//g'"
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
elif system == 'arch':
cmd = "pacman -Qi {} | tr -s ' ' | grep -F 'Version :' | cut -d ' ' -f 3"
else:
raise NotImplementedError(f'_get_package_version not implemented for {system}')
cmd = cmd.format(package)
_, output = execute(cmd, capture=True)
return output.strip()
def require_minimum_package_version(package: str, minimum: str):
"""
Returns true if a given package is available to be installed with
the given minimum version or greater.
:param package: the name of the package that is checked
:param minimum: the semantic version that the package is checked against
"""
version = _get_package_version(package)
if version < minimum:
message = f"ERROR: {package} has version {version}, but must be >= {minimum}"
log.error(message)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
raise UnexpectedError(message)
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
[#3229] hammer.py prepare-system --just-configure
2024-01-23 14:57:08 +02:00
def prepare_system_local(features, check_times, ignore_errors_for, just_configure):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Prepare local system for Kea development based on requested features."""
[#3229] hammer.py prepare-system --just-configure
2024-01-23 14:57:08 +02:00
system, revision = get_system_revision()
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('Preparing deps for %s %s...', system, revision)
[#3229] hammer.py prepare-system --just-configure
2024-01-23 14:57:08 +02:00
if not just_configure:
install_packages_local(system, revision, features, check_times, ignore_errors_for)
if 'mysql' in features:
_configure_mysql(system, revision, features)
if 'pgsql' in features:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
_configure_pgsql(system, features)
[#3229] hammer.py prepare-system --just-configure
2024-01-23 14:57:08 +02:00
log.info('Preparing deps completed successfully.')
def install_packages_local(system, revision, features, check_times, ignore_errors_for):
"""Install packages for Kea development based on requested features."""
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
env = os.environ.copy()
env['LANGUAGE'] = env['LANG'] = env['LC_ALL'] = 'C'
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
# Actions decided before installing packages, but run afterwards
deferred_functions = []
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
# Check if package versions cannot be met.
if 'netconf' in features and 'netconf' not in ignore_errors_for:
require_minimum_package_version('cmake', '3.19')
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare fedora
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if system == 'fedora':
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
packages = ['make', 'autoconf', 'automake', 'libtool', 'gcc-c++', 'openssl-devel',
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
'log4cplus-devel', 'boost-devel', 'libpcap-devel', 'bison', 'flex']
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'native-pkg' in features:
[#1873] hammer: dropped support for python3 in building kea packages
2021-05-17 08:57:57 +02:00
packages.extend(['rpm-build', 'python3-devel'])
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'docs' in features:
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
packages.extend(['python3-sphinx', 'python3-sphinx_rtd_theme',
'texlive', 'texlive-collection-latexextra'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'mysql' in features:
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
packages.extend(['mariadb', 'mariadb-server', 'mariadb-connector-c-devel'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'pgsql' in features:
[#3711] Remove postgresql server dev packages from hammer.py
2025-01-31 11:17:29 +02:00
execute('sudo dnf remove -y postgresql-private-devel', raise_error=False)
packages.extend(['libpq-devel', 'postgresql', 'postgresql-server'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
packages.extend(['krb5-devel'])
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
if 'ccache' in features:
packages.extend(['ccache'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
packages.extend(['cmake', 'git', 'pcre2-devel'])
[#2410] fedora 35 and 35 support
2022-05-18 09:22:15 +02:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'unittest' in features:
[#3350] revert to installing gtest sources for alpine and rpm This is needed to make --with-gtest-source pass in jenkins. There was a false misconception that all gtest packages install sources in /usr/src/googletest. The reality is that alpine and rpm packages install shared libraries instead.
2024-06-05 15:05:04 +03:00
packages.append('wget')
deferred_functions.append(_install_gtest_sources)
[#3350] install googletest packages in hammer
2024-06-03 20:28:16 +03:00
install_pkgs(packages, timeout=300, env=env, check_times=check_times)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
execute('sudo dnf clean packages', env=env, check_times=check_times)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare centos
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
elif system == 'centos':
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
install_pkgs('epel-release', env=env, check_times=check_times)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#2081] install the default openssl-devel on CentOS 7
2021-09-11 19:27:06 +03:00
packages = ['autoconf', 'automake', 'boost-devel', 'gcc-c++',
[#2516] add support for CentOS Stream 9 in hammer.py
2022-07-29 09:51:10 +03:00
'libtool', 'log4cplus-devel', 'make',
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
'openssl-devel', 'bison', 'flex']
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#2539] hammer.py: use venv or packages instead of virtualenv This is in the detriment of virtualenv which is considered deprecated.
2022-09-05 22:30:57 +03:00
if revision in ['7', '8']:
[#2081] install the default openssl-devel on CentOS 7
2021-09-11 19:27:06 +03:00
# Install newer version of Boost in case users want to opt-in with:
# --with-boost-include=/usr/include/boost169 --with-boost-lib-dir=/usr/lib64/boost169
[qa#261] install boost169-devel on CentOS 7
2021-08-23 09:59:02 +03:00
packages.append('boost169-devel')
[qa#261] install OpenSSL 1.1.1g on CentOS 7
2021-08-13 14:03:27 +03:00
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
if 'docs' in features:
packages.extend(['python3-sphinx', 'python3-sphinx_rtd_theme'])
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
if 'native-pkg' in features:
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages.extend(['python3-devel', 'rpm-build'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'mysql' in features:
[#2516] add support for CentOS Stream 9 in hammer.py
2022-07-29 09:51:10 +03:00
packages.extend(['mariadb', 'mariadb-server'])
if int(revision) < 9:
packages.extend(['mariadb-devel'])
else:
packages.extend(['mariadb-connector-c-devel'])
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'pgsql' in features:
[#2539] hammer.py: switch from PostgreSQL 14 to the default in RHEL 9 There is a lot of code that deals with previously installed versions of PostgreSQL. This is good since it deals with a lot of use cases and allows us to not regenerate images, but it also adds complexity.
2022-09-05 21:59:12 +03:00
packages.extend(['postgresql', 'postgresql-server'])
if revision == '9':
packages.append('postgresql13-devel')
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
[#2539] hammer.py: defer linking of pg_config until after package is installed
2022-10-03 10:56:21 +03:00
def link_pg_config():
if not os.path.exists('/usr/bin/pg_config'):
execute('sudo ln -s /usr/pgsql-13/bin/pg_config /usr/bin/pg_config')
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
[#2539] hammer.py: defer linking of pg_config until after package is installed
2022-10-03 10:56:21 +03:00
deferred_functions.append(link_pg_config)
[#2516] add support for CentOS Stream 9 in hammer.py
2022-07-29 09:51:10 +03:00
else:
[#2539] hammer.py: switch from PostgreSQL 14 to the default in RHEL 9 There is a lot of code that deals with previously installed versions of PostgreSQL. This is good since it deals with a lot of use cases and allows us to not regenerate images, but it also adds complexity.
2022-09-05 21:59:12 +03:00
packages.append('postgresql-devel')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
packages.extend(['krb5-devel'])
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
if 'ccache' in features:
packages.extend(['ccache'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
packages.extend(['cmake', 'git', 'pcre2-devel'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'unittest' in features:
[#3350] revert to installing gtest sources for alpine and rpm This is needed to make --with-gtest-source pass in jenkins. There was a false misconception that all gtest packages install sources in /usr/src/googletest. The reality is that alpine and rpm packages install shared libraries instead.
2024-06-05 15:05:04 +03:00
packages.append('wget')
deferred_functions.append(_install_gtest_sources)
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
install_pkgs(packages, env=env, check_times=check_times)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare rhel
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
elif system == 'rhel':
[#2081] install the default openssl-devel on CentOS 7
2021-09-11 19:27:06 +03:00
packages = ['autoconf', 'automake', 'boost-devel', 'gcc-c++',
[#2439] added rhel 9 support to hammer
2022-06-13 18:03:32 +02:00
'libtool', 'log4cplus-devel', 'make',
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
'openssl-devel', 'bison', 'flex']
[qa#261] install OpenSSL 1.1.1g on CentOS 7
2021-08-13 14:03:27 +03:00
Enable additional features in hammer.py for RHEL8
2022-05-13 09:24:28 +02:00
if revision in ['7', '8']:
[#2081] install the default openssl-devel on CentOS 7
2021-09-11 19:27:06 +03:00
# Install newer version of Boost in case users want to opt-in with:
# --with-boost-include=/usr/include/boost169 --with-boost-lib-dir=/usr/lib64/boost169
packages.append('boost169-devel')
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
if 'docs' in features:
# Conflict on the packaging python module between pip and rpm-build.
# So create venv instead of installing through package manager.
# kea-packaging points Kea to the venv using --with-sphinx.
execute('python3 -m venv ~/venv',
env=env, timeout=60, check_times=check_times)
[#3350] upgrade pip in hammer for rhel Fixes this on RHEL 8: You are using pip version 9.0.3, however version 24.0 is available.
2024-06-03 20:16:45 +03:00
execute('~/venv/bin/pip install --upgrade pip',
env=env, timeout=120, check_times=check_times)
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
execute('~/venv/bin/pip install sphinx sphinx-rtd-theme',
env=env, timeout=120, check_times=check_times)
[#2081] install the default openssl-devel on CentOS 7
2021-09-11 19:27:06 +03:00
if 'native-pkg' in features:
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages.extend(['python3-devel', 'rpm-build'])
Enable additional features in hammer.py for RHEL8
2022-05-13 09:24:28 +02:00
Hammer: add MariaDB and PostgreSQL for RHEL 8
2022-05-12 12:34:45 +02:00
if 'mysql' in features:
[#2439] added rhel 9 support to hammer
2022-06-13 18:03:32 +02:00
packages.extend(['mariadb', 'mariadb-server'])
if int(revision) < 9:
packages.extend(['mariadb-devel'])
else:
packages.extend(['mariadb-connector-c-devel'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
Hammer: add MariaDB and PostgreSQL for RHEL 8
2022-05-12 12:34:45 +02:00
if 'pgsql' in features:
[#3711] hammer.py: Switch to libpq in RHEL too
2025-02-05 16:17:15 +02:00
execute('sudo dnf remove -y postgresql-private-devel', raise_error=False)
packages.extend(['libpq-devel', 'postgresql', 'postgresql-server'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
Enable additional features in hammer.py for RHEL8
2022-05-13 09:24:28 +02:00
if 'gssapi' in features:
packages.extend(['krb5-devel'])
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
if 'ccache' in features:
packages.extend(['ccache'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
packages.extend(['cmake', 'git', 'pcre2-devel'])
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'unittest' in features:
[#3350] revert to installing gtest sources for alpine and rpm This is needed to make --with-gtest-source pass in jenkins. There was a false misconception that all gtest packages install sources in /usr/src/googletest. The reality is that alpine and rpm packages install shared libraries instead.
2024-06-05 15:05:04 +03:00
packages.append('wget')
deferred_functions.append(_install_gtest_sources)
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
install_pkgs(packages, env=env, timeout=120, check_times=check_times)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
# prepare rocky
elif system == 'rocky':
install_pkgs('epel-release', env=env, check_times=check_times)
packages = ['autoconf', 'automake', 'bison', 'boost-devel', 'flex', 'gcc-c++',
'libtool', 'log4cplus-devel', 'make',
'openssl-devel']
if 'docs' in features:
packages.extend(['python3-sphinx', 'python3-sphinx_rtd_theme'])
if 'native-pkg' in features:
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages.extend(['python3-devel', 'rpm-build'])
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if 'mysql' in features:
packages.extend(['mariadb', 'mariadb-server', 'mariadb-connector-c-devel'])
if 'pgsql' in features:
[#3711] Remove postgresql server dev packages from hammer.py
2025-01-31 11:17:29 +02:00
packages.extend(['libpq-devel', 'postgresql', 'postgresql-server'])
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if 'gssapi' in features:
packages.extend(['krb5-devel'])
if 'ccache' in features:
packages.extend(['ccache'])
if 'netconf' in features:
packages.extend(['cmake', 'git', 'pcre2-devel'])
if 'unittest' in features:
[#3350] revert to installing gtest sources for alpine and rpm This is needed to make --with-gtest-source pass in jenkins. There was a false misconception that all gtest packages install sources in /usr/src/googletest. The reality is that alpine and rpm packages install shared libraries instead.
2024-06-05 15:05:04 +03:00
packages.append('wget')
deferred_functions.append(_install_gtest_sources)
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
execute('sudo dnf config-manager --set-enabled crb')
install_pkgs(packages, env=env, timeout=120, check_times=check_times)
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare ubuntu
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif system == 'ubuntu':
hammer: fixed building on debian buster
2019-07-22 10:52:40 +02:00
_apt_update(system, revision, env=env, check_times=check_times, attempts=3, sleep_time_after_attempt=10)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
packages = ['gcc', 'g++', 'make', 'autoconf', 'automake', 'libtool', 'libssl-dev', 'liblog4cplus-dev',
[#3393] ignore error in libyang tests ... and add bison and flex to ubuntu for completion
2024-05-17 08:47:55 +03:00
'libboost-system-dev', 'gnupg', 'libpcap-dev', 'bison', 'flex']
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
if 'docs' in features:
packages.extend(['python3-sphinx', 'python3-sphinx-rtd-theme',
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
'texlive', 'texlive-latex-extra', 'tex-gyre'])
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'unittest' in features:
[#3350] install googletest packages in hammer
2024-06-03 20:28:16 +03:00
packages.append('googletest')
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'native-pkg' in features:
packages.extend(['build-essential', 'fakeroot', 'devscripts'])
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages.extend(['debhelper', 'libboost-dev', 'python3-dev'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 20.04 <= float(revision):
[#1229] hammer: added support for building kea on fedora 32 and ubuntu 20.04
2020-05-12 11:39:17 +02:00
packages.extend(['dh-python'])
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'mysql' in features:
[#3382] change to mariadb in ubuntu 24
2024-05-14 08:28:58 +00:00
if int(revision.split(".")[0]) < 24:
if revision == '16.04':
packages.extend(['mysql-client', 'libmysqlclient-dev', 'mysql-server'])
else:
packages.extend(['default-mysql-client-core', 'default-libmysqlclient-dev', 'mysql-server'])
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
else:
[#3382] change to mariadb in ubuntu 24
2024-05-14 08:28:58 +00:00
packages.extend(['mariadb-client', 'mariadb-server', 'libmariadb-dev-compat'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'pgsql' in features:
[#3711] Remove postgresql server dev packages from hammer.py
2025-01-31 11:17:29 +02:00
packages.extend(['libpq-dev', 'postgresql', 'postgresql-client'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
packages.extend(['libkrb5-dev'])
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
if 'ccache' in features:
packages.extend(['ccache'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
packages.extend(['cmake', 'git', 'libpcre2-dev'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
install_pkgs(packages, env=env, timeout=240, check_times=check_times)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare debian
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif system == 'debian':
hammer: fixed building on debian buster
2019-07-22 10:52:40 +02:00
_apt_update(system, revision, env=env, check_times=check_times, attempts=3, sleep_time_after_attempt=10)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
packages = ['gcc', 'g++', 'make', 'autoconf', 'automake', 'libtool', 'libssl-dev',
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
'liblog4cplus-dev', 'libboost-system-dev', 'gnupg', 'bison', 'flex']
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
if 'docs' in features:
[#3350] clean up some old revision logic in hammer
2024-06-03 18:00:25 +03:00
packages.extend(['python3-sphinx', 'python3-sphinx-rtd-theme', 'doxygen', 'graphviz',
[#2969] add tex-gyre back to hammer for debian tex-gyre is needed for generating PDF with sphinx.
2023-11-13 11:49:32 +02:00
'tex-gyre', 'texlive', 'texlive-latex-extra'])
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'unittest' in features:
[#3350] install googletest packages in hammer
2024-06-03 20:28:16 +03:00
packages.append('googletest')
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3350] clean up some old revision logic in hammer
2024-06-03 18:00:25 +03:00
packages.extend(['cmake', 'git', 'libpcre2-dev', 'pkg-config'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
if 'native-pkg' in features:
packages.extend(['build-essential', 'fakeroot', 'devscripts'])
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages.extend(['debhelper', 'libboost-dev', 'python3-dev'])
[#2193] added dh-python on debian 11
2021-11-18 13:17:07 +01:00
if int(revision) >= 11:
packages.extend(['dh-python'])
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'mysql' in features:
[#3350] clean up some old revision logic in hammer
2024-06-03 18:00:25 +03:00
packages.extend(['default-mysql-client-core', 'default-libmysqlclient-dev'])
if int(revision) <= 8:
packages.extend(['mysql-client', 'libmysqlclient-dev', 'mysql-server'])
elif int(revision) <= 9:
hammer: added support for debian-10, some other various fixes
2019-06-24 21:58:48 +02:00
packages.append('mysql-server')
else:
packages.append('mariadb-server')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
if 'pgsql' in features:
[#3711] Remove postgresql server dev packages from hammer.py
2025-01-31 11:17:29 +02:00
packages.extend(['libpq-dev', 'postgresql', 'postgresql-client'])
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
packages.extend(['libkrb5-dev'])
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
if 'ccache' in features:
packages.extend(['ccache'])
[#372,!181] hammer: hardened running builds, improved building with mysql support
2019-01-29 14:46:07 +01:00
install_pkgs(packages, env=env, timeout=240, check_times=check_times)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
# prepare freebsd
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif system == 'freebsd':
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages = ['autoconf', 'automake', 'bison', 'flex', 'libtool', 'openssl', 'log4cplus', 'boost-libs', 'wget']
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
if revision.startswith('14'):
packages.extend(['bash', 'pkgconf'])
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if 'docs' in features:
[#2534] Dynamic python ver detection patch by Andrei Pavel
2022-08-26 09:48:13 +02:00
# Get the python version from the remote repositories.
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
pyv = _get_package_version('python')
pyv = pyv.split('_')[0].replace('.', '')
[#2534] Added log about detected python ver
2022-08-26 09:59:08 +02:00
log.info(">>>>> Detected Sphinx packages version: py%s-sphinx", pyv)
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
packages.extend([f'py{pyv}-sphinx', f'py{pyv}-sphinx_rtd_theme'])
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
if 'mysql' in features:
[#1658] addressed review issues
2021-06-08 16:03:51 +02:00
if revision.startswith(('11', '12')):
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
packages.extend(['mysql57-server', 'mysql57-client'])
[#1658] addressed review issues
2021-06-08 16:03:51 +02:00
else:
packages.extend(['mysql80-server', 'mysql80-client'])
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
Added support for PostgreSQL DB on FreeBSD systems
2019-08-10 21:47:38 +02:00
if 'pgsql' in features:
[#3470] hammer: install postgresql only if not already installed
2024-08-08 10:46:46 +03:00
# Install the latest postgresql-client and postgresql-server,
# unless any postgresql-client or postgresql-server version is already installed.
for i in ['client', 'server']:
# Check if already installed.
_, output = execute('pkg info', capture=True)
m = re.search(f'postgresql[0-9]+-{i}', output)
if m is None:
# If not, go ahead and install.
_, output = execute('pkg search postgresql', capture=True)
found = re.findall(f'postgresql[0-9]+-{i}', output)
if len(found) == 0:
print(f'No postgresql[0-9]+-{i} found?')
sys.exit(1)
# There may be more matches. Results are sorted by pkg.
# Choose the last from the list which should be the latest version.
packages.append(found[-1])
Added support for PostgreSQL DB on FreeBSD systems
2019-08-10 21:47:38 +02:00
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
packages.extend(['krb5-devel'])
[#2539] hammer.py: delete the default Heimdal krb5-config in FreeBSD This makes it possible for Kea to pick up by default on the krb5-config from the MIT version of Kerberos packages.
2022-09-05 22:26:15 +03:00
# FreeBSD comes with a Heimdal krb5-config by default. Make sure
# it's deleted so that Kea uses the MIT packages added just above.
execute('sudo rm -f /usr/bin/krb5-config')
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
if 'ccache' in features:
packages.extend(['ccache'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
packages.extend(['cmake', 'git', 'pcre2'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
if 'unittest' in features:
[#3698] Install gtest sources on FreeBSD Fixes "checking for gtest source... configure: error: no gtest source at /usr/src/googletest".
2024-12-19 10:56:37 +02:00
deferred_functions.append(_install_gtest_sources)
[#3350] revert to installing gtest sources for alpine and rpm This is needed to make --with-gtest-source pass in jenkins. There was a false misconception that all gtest packages install sources in /usr/src/googletest. The reality is that alpine and rpm packages install shared libraries instead.
2024-06-05 15:05:04 +03:00
install_pkgs(packages, env=env, timeout=6 * 60, check_times=check_times)
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
if 'mysql' in features:
execute('sudo sysrc mysql_enable="yes"', env=env, check_times=check_times)
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
execute('sudo service mysql-server start', env=env, check_times=check_times,
raise_error=False)
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
execute('sudo pkg clean -a -y')
execute('sudo rm -rf /usr/lib/debug')
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# prepare alpine
elif system == 'alpine':
packages = ['gcc', 'g++', 'make', 'autoconf', 'automake', 'libtool', 'openssl-dev',
[#2503] Ensure Alpine has real gzip installed
2022-07-22 01:27:39 -07:00
'boost-libs', 'boost-dev', 'procps', 'tar', 'log4cplus', 'log4cplus-dev',
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
'gzip', 'bison', 'flex']
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if 'docs' in features:
[#2969] remove sphinx-tabs from hammer.py - use package managers to install sphinx where possible - simplify the logic in some areas
2023-11-03 16:51:33 +02:00
packages.extend(['py3-sphinx py3-sphinx_rtd_theme'])
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#3445] hammer: add git and change bash to sh
2024-06-03 16:38:43 +03:00
packages.extend(['cmake', 'git', 'pcre2-dev'])
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if 'mysql' in features:
packages.extend(['mariadb-dev', 'mariadb', 'mariadb-client'])
if 'pgsql' in features:
packages.extend(['postgresql-dev', 'postgresql'])
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
packages.extend(['krb5-dev'])
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if 'native-pkg' in features:
[#3350] put bison and flex as free packages in hammer
2024-06-03 17:52:32 +03:00
packages.extend(['alpine-sdk', 'python3-dev'])
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
hammer: fixed ccache for alpine
2019-10-01 06:12:53 +02:00
if 'ccache' in features:
packages.extend(['ccache'])
[#3350] revert to installing gtest sources for alpine and rpm This is needed to make --with-gtest-source pass in jenkins. There was a false misconception that all gtest packages install sources in /usr/src/googletest. The reality is that alpine and rpm packages install shared libraries instead.
2024-06-05 15:05:04 +03:00
if 'unittest' in features:
packages.append('wget')
deferred_functions.append(_install_gtest_sources)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
install_pkgs(packages, env=env, timeout=6 * 60, check_times=check_times)
Fixed adding vagrant user to abuild group on Alpine
2020-10-20 14:58:03 +02:00
# check for existence of 'vagrant' user and 'abuild' group before adding him to the group
try:
pwd.getpwnam('vagrant')
grp.getgrnam('abuild')
except KeyError:
log.info("Can't add 'vagrant' user to 'abuild' group. Vagrant or abuild does not exist.")
else:
execute('sudo adduser vagrant abuild')
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
try:
current_user = getpass.getuser()
pwd.getpwnam(current_user)
grp.getgrnam('abuild')
except KeyError:
log.info("Can't add %s user to 'abuild' group. %s or abuild does not exist.", current_user, current_user)
else:
execute('sudo adduser %s abuild' % current_user)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
else:
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
raise NotImplementedError('no implementation for %s' % system)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
if 'netconf' in features:
_install_netconf_libraries_from_sources('netconf' in ignore_errors_for)
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
# Packages required by these functions have been installed. Now call them.
for f in deferred_functions:
f()
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
def prepare_system_in_vagrant(provider, system, revision, features, dry_run, check_times,
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
clean_start, ccache_dir=None):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Prepare specified system in Vagrant according to specified features."""
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
ve = VagrantEnv(provider, system, revision, features, 'kea', dry_run, check_times=check_times,
ccache_dir=ccache_dir)
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
if clean_start:
ve.destroy()
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
ve.init_files()
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
ve.bring_up_latest_box()
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
ve.prepare_system()
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
def _calculate_build_timeout(features):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Return maximum allowed time for build (in seconds)."""
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
timeout = 60
if 'mysql' in features:
timeout += 60
timeout *= 60
return timeout
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
def _prepare_ccache_if_needed(system, ccache_dir, env):
if ccache_dir is not None:
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
if system in ['debian', 'ubuntu']:
ccache_bin_path = '/usr/lib/ccache/'
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
elif system in ['centos', 'rhel', 'fedora', 'rocky']:
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
ccache_bin_path = '/usr/lib64/ccache'
env['CC'] = 'ccache gcc'
env['CXX'] = 'ccache g++'
hammer: fixed ccache for alpine
2019-10-01 06:12:53 +02:00
elif system == 'alpine':
# TODO: it doesn't work yet, new abuild is needed and add 'USE_CCACHE=1' to /etc/abuild.conf
ccache_bin_path = '/usr/lib/ccache/bin'
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
env['PATH'] = ccache_bin_path + ':' + env['PATH']
env['CCACHE_DIR'] = ccache_dir
return env
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
def _build_binaries_and_run_ut(system, revision, features, tarball_path, env, check_times, jobs, dry_run,
ccache_dir):
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if tarball_path:
# unpack tarball with sources
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
execute('sudo rm -rf kea-src')
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
os.mkdir('kea-src')
execute('tar -zxf %s' % tarball_path, cwd='kea-src', check_times=check_times)
src_path = glob.glob('kea-src/*')[0]
else:
src_path = '.'
execute('autoreconf -f -i', cwd=src_path, env=env, dry_run=dry_run)
# prepare switches for ./configure
cmd = './configure'
[#934,!531] added support for centos8 rpm
2019-10-14 19:03:46 +07:00
log.info('OS: %s Revision: %s', system, revision)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if 'mysql' in features:
cmd += ' --with-mysql'
if 'pgsql' in features:
cmd += ' --with-pgsql'
if 'unittest' in features:
[#3350] clean up some old revision logic in hammer
2024-06-03 18:00:25 +03:00
cmd += ' --with-gtest-source=/usr/src/googletest'
[#2439] added rhel 9 support to hammer
2022-06-13 18:03:32 +02:00
if 'docs' in features and not system == 'rhel':
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
cmd += ' --enable-generate-docs'
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
if 'gssapi' in features:
[qa#276] fix hammer's netconf and postgres actions on Fedora 33 and FreeBSDs
2021-11-10 21:04:19 +02:00
cmd += ' --with-gssapi'
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if 'shell' in features:
cmd += ' --enable-shell'
perfdhcp avalache: improvements after review - simplified differentiating for IP version and exchange types - added more comments - fixed unittest for receiver by mocking socket - added option for building perfdhcp by hammer - added workaround for compiler bug that cannot handle enum class as a key to std::unordered_map - hidden warnings from boost by changing compiler flag from -I<boost-path> to -isystem <boost-path> - removed unused options_ field from StatsMgr class
2019-02-19 14:37:00 +01:00
if 'perfdhcp' in features:
cmd += ' --enable-perfdhcp'
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
cmd += ' --with-libyang --with-sysrepo --with-libyang-cpp --with-sysrepo-cpp'
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# do ./configure
execute(cmd, cwd=src_path, env=env, timeout=120, check_times=check_times, dry_run=dry_run)
[#1993] hammer.py: support for NETCONF installs libyang and sysrepo
2021-08-05 18:12:10 +03:00
if 'netconf' in features:
# ./configure has created reinstall.sh from reinstall.sh.in. Call it.
execute('./src/share/yang/modules/utils/reinstall.sh', cwd=src_path, env=env)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# estimate number of processes (jobs) to use in compilation if jobs are not provided
if jobs == 0:
cpus = multiprocessing.cpu_count() - 1
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
if system == 'centos':
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
cpus = cpus // 2
if cpus == 0:
cpus = 1
else:
cpus = jobs
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
# enable ccache if requested
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
env = _prepare_ccache_if_needed(system, ccache_dir, env)
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# do build
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
timeout = _calculate_build_timeout(features)
if 'distcheck' in features:
cmd = 'make distcheck'
else:
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
cmd = 'make -j%s' % cpus
[#372,!181] hammer: added distcheck feature, fixed mysql in freebsd
2019-01-30 18:11:18 +01:00
execute(cmd, cwd=src_path, env=env, timeout=timeout, check_times=check_times, dry_run=dry_run)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if 'unittest' in features:
results_dir = os.path.abspath(os.path.join(src_path, 'tests_result'))
execute('rm -rf %s' % results_dir, dry_run=dry_run)
if not os.path.exists(results_dir):
os.mkdir(results_dir)
env['GTEST_OUTPUT'] = 'xml:%s/' % results_dir
env['KEA_SOCKET_TEST_DIR'] = '/tmp/'
# run unit tests
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
execute('make check -k',
perfdhcp avalache: improvements after review - simplified differentiating for IP version and exchange types - added more comments - fixed unittest for receiver by mocking socket - added option for building perfdhcp by hammer - added workaround for compiler bug that cannot handle enum class as a key to std::unordered_map - hidden warnings from boost by changing compiler flag from -I<boost-path> to -isystem <boost-path> - removed unused options_ field from StatsMgr class
2019-02-19 14:37:00 +01:00
cwd=src_path, env=env, timeout=90 * 60, raise_error=False,
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
check_times=check_times, dry_run=dry_run)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# parse unit tests results
results = {}
grand_total = 0
grand_not_passed = 0
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
aggregated_root = ET.Element('testsuites')
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
for fn in os.listdir(results_dir):
if not fn.endswith('.xml'):
continue
fp = os.path.join(results_dir, fn)
tree = ET.parse(fp)
root = tree.getroot()
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
# prepare stats for json
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
total = int(root.get('tests'))
failures = int(root.get('failures'))
disabled = int(root.get('disabled'))
errors = int(root.get('errors'))
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
results[fn] = {
'total': total,
'failures': failures,
'disabled': disabled,
'errors': errors,
}
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
grand_total += total
grand_not_passed += failures + errors
[#1760] spell checks
2021-03-18 10:22:04 +02:00
# append test suites to aggregated root
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
for ts in root.findall('testsuite'):
if not ts:
continue
aggregated_root.append(ts)
# prepare and stats in json
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
grand_passed = grand_total - grand_not_passed
results['grand_passed'] = grand_total - grand_not_passed
results['grand_total'] = grand_total
result = '%s/%s passed' % (grand_passed, grand_total)
if grand_not_passed > 0 or grand_total == 0:
result = red(result)
else:
result = green(result)
log.info('Unit test results: %s', result)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
with open('unit-test-results.json', 'w', encoding='utf-8') as f:
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
f.write(json.dumps(results))
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
# store aggregated results in XML
if os.path.exists('aggregated_tests.xml'):
os.unlink('aggregated_tests.xml')
aggr = ET.ElementTree(aggregated_root)
aggr.write('aggregated_tests.xml')
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if 'install' in features:
perfdhcp avalache: improvements after review - simplified differentiating for IP version and exchange types - added more comments - fixed unittest for receiver by mocking socket - added option for building perfdhcp by hammer - added workaround for compiler bug that cannot handle enum class as a key to std::unordered_map - hidden warnings from boost by changing compiler flag from -I<boost-path> to -isystem <boost-path> - removed unused options_ field from StatsMgr class
2019-02-19 14:37:00 +01:00
execute('sudo make install', timeout=2 * 60,
cwd=src_path, env=env, check_times=check_times, dry_run=dry_run)
[#2539] hammer.py: ldconfig after make install
2022-10-13 14:04:25 +03:00
if system != 'alpine':
execute('sudo ldconfig', dry_run=dry_run, env=env)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if 'forge' in features:
if 'mysql' in features:
[#871,!497] hammer: switch from from kea-admin lease-init to db-init
2019-08-30 17:13:04 +02:00
execute('kea-admin db-init mysql -u keauser -p keapass -n keadb', dry_run=dry_run)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if 'pgsql' in features:
[#871,!497] hammer: switch from from kea-admin lease-init to db-init
2019-08-30 17:13:04 +02:00
execute('kea-admin db-init pgsql -u keauser -p keapass -n keadb', dry_run=dry_run)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
def _check_installed_rpm_or_debs(services_list):
for svc in services_list:
[kea-packaging#8] show logs when _STARTED Kea not found Also make reading of logs more pythonic.
2024-08-12 19:56:02 +03:00
execute(f'sudo systemctl stop {svc}')
[#1262] hammer: hardened checking installed packages
2020-05-29 15:25:16 +02:00
now = datetime.datetime.now()
timestamp = now.strftime('%Y-%m-%d%H:%M:%S')
[kea-packaging#8] show logs when _STARTED Kea not found Also make reading of logs more pythonic.
2024-08-12 19:56:02 +03:00
execute(f'sudo systemctl start {svc}')
logs = ''
for _ in range(10):
_, logs = execute(f'sudo journalctl --since {timestamp} -u {svc}', capture=True)
if '_STARTED Kea' in logs:
break
time.sleep(1)
if '_STARTED Kea' not in logs:
print(logs)
raise UnexpectedError('_STARTED Kea not in logs')
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
def _build_rpm(system, revision, features, tarball_path, env, check_times, dry_run,
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_version, pkg_isc_version):
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# unpack kea sources tarball
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
_, arch = execute('arch', capture=True)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute('sudo rm -rf kea-src', dry_run=dry_run)
os.mkdir('kea-src')
execute('tar -zxf %s' % tarball_path, cwd='kea-src', check_times=check_times, dry_run=dry_run)
src_path = glob.glob('kea-src/*')[0]
# prepare folder for all pkgs
if os.path.exists('pkgs'):
execute('rm -rf pkgs')
os.mkdir('pkgs')
# prepare RPM environment
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
rpm_root_path = os.path.expanduser('~/rpm-root')
# ensure rm -rf will not wipe out a whole disk
if rpm_root_path.endswith("rpm-root"):
execute('rm -rf %s' % rpm_root_path)
execute('mkdir -p %s/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}' % rpm_root_path)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# get rpm.spec from tarball
rpm_dir = os.path.join(src_path, 'rpm')
for f in os.listdir(rpm_dir):
if f == 'kea.spec':
continue
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute('cp %s %s/SOURCES' % (os.path.join(rpm_dir, f), rpm_root_path), check_times=check_times,
dry_run=dry_run)
execute('cp %s %s/SPECS' % (os.path.join(rpm_dir, 'kea.spec'), rpm_root_path), check_times=check_times,
dry_run=dry_run)
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
execute('cp %s %s/SOURCES' % (tarball_path, rpm_root_path), check_times=check_times, dry_run=dry_run)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[#3060] fix for centos 7 pkgs, updated deb services names
2023-09-21 11:23:12 +00:00
services_list = ['kea-dhcp4.service', 'kea-dhcp6.service', 'kea-dhcp-ddns.service', 'kea-ctrl-agent.service']
# centos/rhel 7 does not support some fields in systemd unit files so they need to be commented out
if system == 'centos' and revision == '7':
for f in services_list:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
for k in ['RuntimeDirectory', 'RuntimeDirectoryPreserve', 'LogsDirectory', 'LogsDirectoryMode',
'StateDirectory', 'ConfigurationDirectory']:
cmd = r"sed -i -E 's/^(%s=.*)/#\1/' %s" % (k, f)
[#3060] fix for centos 7 pkgs, updated deb services names
2023-09-21 11:23:12 +00:00
execute(cmd, cwd=rpm_dir, check_times=check_times, dry_run=dry_run)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# do rpm build
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
cmd = "rpmbuild --define 'kea_version %s' --define 'isc_version %s' -ba %s/SPECS/kea.spec"
cmd += " -D'_topdir %s'"
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
cmd += " --undefine=_debugsource_packages" # disable creating debugsource package
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
cmd = cmd % (pkg_version, pkg_isc_version, rpm_root_path, rpm_root_path)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute(cmd, env=env, timeout=60 * 40, check_times=check_times, dry_run=dry_run)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if 'install' in features:
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# install packages
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute('rpm -qa | grep isc-kea | xargs sudo rpm -e', check_times=check_times, dry_run=dry_run,
raise_error=False)
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
execute(f'sudo rpm -i {rpm_root_path}/RPMS/{arch.strip()}/*rpm', check_times=check_times, dry_run=dry_run)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# check if kea services can be started
services_list = ['kea-dhcp4.service', 'kea-dhcp6.service', 'kea-dhcp-ddns.service', 'kea-ctrl-agent.service']
_check_installed_rpm_or_debs(services_list)
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
execute(f'mv {rpm_root_path}/RPMS/{arch.strip()}/*rpm pkgs', check_times=check_times, dry_run=dry_run)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
def _build_deb(system, revision, features, tarball_path, env, check_times, dry_run,
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_version, pkg_isc_version, repo_url):
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
_, arch = execute('arch', capture=True)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if system == 'debian' and revision == '9':
# debian 9 does not support apt-installing over https, so install proper transport
install_pkgs('apt-transport-https', env=env, check_times=check_times)
[#2539] hammer.py: warn if there is no Release file It solves errors that appear on the first time a Debian package is uploaded like: E: The repository 'https://packages.aws.isc.org/repository/kea-2.3-debian-11-ci kea Release' does not have a Release file.
2022-09-06 16:38:44 +03:00
# See if a .deb package had been previously uploaded.
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
_, output = execute("curl -o /dev/null -s -w '%{{http_code}}' {}/dists/kea/Release 2>/dev/null".format(repo_url),
capture=True)
[#2539] hammer.py: warn if there is no Release file It solves errors that appear on the first time a Debian package is uploaded like: E: The repository 'https://packages.aws.isc.org/repository/kea-2.3-debian-11-ci kea Release' does not have a Release file.
2022-09-06 16:38:44 +03:00
http_code = output.rstrip()
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
release_file_exists = http_code == '200'
[#2539] hammer.py: warn if there is no Release file It solves errors that appear on the first time a Debian package is uploaded like: E: The repository 'https://packages.aws.isc.org/repository/kea-2.3-debian-11-ci kea Release' does not have a Release file.
2022-09-06 16:38:44 +03:00
if release_file_exists:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('%s/dists/kea/Release exists.', repo_url)
[#2539] hammer.py: warn if there is no Release file It solves errors that appear on the first time a Debian package is uploaded like: E: The repository 'https://packages.aws.isc.org/repository/kea-2.3-debian-11-ci kea Release' does not have a Release file.
2022-09-06 16:38:44 +03:00
else:
repo_name = 'kea-%s-%s-%s' % (pkg_version.rsplit('.', 1)[0], system, revision)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.error('%s/dists/kea/Release does not exist. '
'This is usually caused by no package existing in %s. '
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
'You can solve this by uploading any package.'
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
'Continuing, but the build will likely fail.', repo_url, repo_name)
[#2539] hammer.py: warn if there is no Release file It solves errors that appear on the first time a Debian package is uploaded like: E: The repository 'https://packages.aws.isc.org/repository/kea-2.3-debian-11-ci kea Release' does not have a Release file.
2022-09-06 16:38:44 +03:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# try apt update for up to 10 times if there is an error
for _ in range(10):
_, out = _apt_update(system, revision, capture=True)
if 'Bad header data' not in out:
break
time.sleep(4)
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# unpack tarball
execute('sudo rm -rf kea-src', check_times=check_times, dry_run=dry_run)
os.mkdir('kea-src')
execute('tar -zxf %s' % tarball_path, cwd='kea-src', check_times=check_times, dry_run=dry_run)
src_path = glob.glob('kea-src/*')[0]
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# update version, etc
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
execute('sed -i -e s/{VERSION}/%s/ changelog' % pkg_version, cwd='kea-src/kea-%s/debian' % pkg_version,
check_times=check_times, dry_run=dry_run)
execute('sed -i -e s/{ISC_VERSION}/%s/ changelog' % pkg_isc_version, cwd='kea-src/kea-%s/debian' % pkg_version,
check_times=check_times, dry_run=dry_run)
execute('sed -i -e s/{ISC_VERSION}/%s/ rules' % pkg_isc_version, cwd='kea-src/kea-%s/debian' % pkg_version,
check_times=check_times, dry_run=dry_run)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
services_list = ['isc-kea-dhcp4.isc-kea-dhcp4-server.service', 'isc-kea-dhcp6.isc-kea-dhcp6-server.service',
'isc-kea-dhcp-ddns.isc-kea-dhcp-ddns-server.service', 'isc-kea-ctrl-agent.service']
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# debian 9 does not support some fields in systemd unit files so they need to be commented out
if system == 'debian' and revision == '9':
for f in services_list:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
for k in ['RuntimeDirectory', 'RuntimeDirectoryPreserve', 'LogsDirectory', 'LogsDirectoryMode',
'StateDirectory', 'ConfigurationDirectory']:
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
cmd = "sed -i -E 's/^(%s=.*)/#\\1/' %s" % (k, f)
execute(cmd, cwd='kea-src/kea-%s/debian' % pkg_version, check_times=check_times, dry_run=dry_run)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# do deb build
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
env['LIBRARY_PATH'] = f'/usr/lib/{arch.strip()}-linux-gnu'
env['LD_LIBRARY_PATH'] = f'/usr/lib/{arch.strip()}-linux-gnu'
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
cmd = ('debuild --preserve-envvar=LD_LIBRARY_PATH --preserve-envvar=LIBRARY_PATH --preserve-envvar=CCACHE_DIR '
'--prepend-path=/usr/lib/ccache -i -us -uc -b')
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute(cmd, env=env, cwd=src_path, timeout=60 * 40, check_times=check_times, dry_run=dry_run)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if 'install' in features:
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# install packages
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute('sudo dpkg -i kea-src/*deb', check_times=check_times, dry_run=dry_run)
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# check if kea services can be started
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
services_list = ['isc-kea-dhcp4-server.service', 'isc-kea-dhcp6-server.service',
'isc-kea-dhcp-ddns-server.service', 'isc-kea-ctrl-agent.service']
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
_check_installed_rpm_or_debs(services_list)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
def _build_alpine_apk(revision, features, tarball_path, check_times, dry_run,
pkg_version, pkg_isc_version):
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
_, arch = execute('arch', capture=True)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
# unpack tarball
execute('sudo rm -rf kea-src packages', check_times=check_times, dry_run=dry_run)
os.makedirs('kea-src/src')
execute('tar -zxf %s' % tarball_path, check_times=check_times, dry_run=dry_run)
[#1229] hammer: added support for building kea on alpine 3.11
2020-05-15 07:36:04 +02:00
if revision == '3.10':
sys_suffix = '-3.10'
else:
sys_suffix = ''
execute('mv kea-%s/alpine%s/* kea-src' % (pkg_version, sys_suffix), check_times=check_times, dry_run=dry_run)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute('rm -rf kea-%s' % pkg_version, check_times=check_times, dry_run=dry_run)
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
tardir = os.path.dirname(tarball_path)
if not tardir:
tardir = '.'
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
cmd = ('cd %s; export kea_chks=`sha512sum kea-%s.tar.gz`; cd -; '
'sed -i -e "s/KEA_CHECKSUM/${kea_chks}/" kea-src/APKBUILD' % (tardir, pkg_version))
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute(cmd, check_times=check_times, dry_run=dry_run)
cmd = 'sed -i -e s/KEA_VERSION/%s/ kea-src/APKBUILD' % pkg_version
execute(cmd, check_times=check_times, dry_run=dry_run)
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
cmd = 'sed -i -e s/KEA_ISC_VERSION/%s/ kea-src/APKBUILD' % pkg_isc_version[3:]
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute(cmd, check_times=check_times, dry_run=dry_run)
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
log.info('Moving %s to kea-src folder', tarball_path)
execute('mv %s kea-src' % tarball_path, check_times=check_times, dry_run=dry_run)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
execute('abuild-keygen -n -a -i', check_times=check_times, dry_run=dry_run)
execute('abuild -v -r', cwd='kea-src', check_times=check_times, dry_run=dry_run)
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
[#1921] hammer: fixed building on alpine, ubuntu, debian and freebsd
2021-06-09 13:03:07 +02:00
# copy packages from alpine specific dir with produced pkgs to common place
alpine_repo_dir = os.path.basename(os.getcwd())
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
src_dir = f'~/packages/{alpine_repo_dir}/{arch.strip()}'
execute(f'cp {src_dir}/*.apk kea-pkg', check_times=check_times, dry_run=dry_run)
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
if 'install' in features:
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# install packages
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
execute('sudo apk add *.apk', cwd='kea-pkg', check_times=check_times, dry_run=dry_run)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
# check if kea services can be started
for svc in ['kea-dhcp4', 'kea-dhcp6', 'kea-ctrl-agent', 'kea-dhcp-ddns']:
[kea-packaging#8] show logs when _STARTED Kea not found Also make reading of logs more pythonic.
2024-08-12 19:56:02 +03:00
execute(f'sudo rc-service {svc} start')
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
if svc == 'kea-dhcp-ddns':
svc = 'kea-ddns'
[kea-packaging#8] show logs when _STARTED Kea not found Also make reading of logs more pythonic.
2024-08-12 19:56:02 +03:00
logs = ''
log_path = f'/var/log/kea/{svc}.log'
for _ in range(10):
_, logs = execute(f'sudo cat {log_path}', capture=True)
if '_STARTED Kea' in logs:
break
time.sleep(1)
if '_STARTED Kea' not in logs:
print(logs)
raise UnexpectedError('_STARTED Kea not in logs')
[#1261] hammer: fixed packaging for debian 9 - added removing unsupported fields for systemd unit files - added checking built and installed kea services if they can start
2020-05-29 07:47:04 +02:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
def _build_native_pkg(system, revision, features, tarball_path, env, check_times, dry_run, ccache_dir,
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
pkg_version, pkg_isc_version, repository_url, pkgs_dir):
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
"""Build native (RPM or DEB or Alpine APK) packages."""
# enable ccache if requested
env = _prepare_ccache_if_needed(system, ccache_dir, env)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
repo_url = _get_full_repo_url(repository_url, system, revision)
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
if repo_url is None:
raise ValueError('repo_url is None')
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if system in ['fedora', 'centos', 'rhel', 'rocky']:
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
_build_rpm(system, revision, features, tarball_path, env, check_times, dry_run,
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_version, pkg_isc_version)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
elif system in ['ubuntu', 'debian']:
_build_deb(system, revision, features, tarball_path, env, check_times, dry_run,
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
pkg_version, pkg_isc_version, repo_url)
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
elif system in ['alpine']:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
_build_alpine_apk(revision, features, tarball_path, check_times, dry_run,
pkg_version, pkg_isc_version)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
elif system in ['arch']:
pass
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
else:
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
raise NotImplementedError('no implementation for %s' % system)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
if system in ['ubuntu', 'debian']:
execute('mv kea-src/isc-kea_* %s' % pkgs_dir)
execute('mv kea-src/*deb %s' % pkgs_dir)
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
elif system in ['fedora', 'centos', 'rhel', 'rocky']:
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
execute('mv pkgs/* %s' % pkgs_dir)
elif system in ['alpine']:
[#3147] trivial hammer fix: don't move files Don't move files to themselves. Solves error: mv: 'kea-pkg/isc-kea-2.5.4-r20231113065823.apk' and 'kea-pkg/isc-kea-2.5.4-r20231113065823.apk' are the same file
2023-11-13 12:04:34 +02:00
# Don't move files if the source and the target locations are the same.
if pkgs_dir != 'kea-pkg':
execute('mv kea-pkg/* %s' % pkgs_dir)
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
elif system in ['arch']:
pass
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
else:
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
raise NotImplementedError('no implementation for %s' % system)
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
def build_local(features, tarball_path, check_times, jobs, dry_run, ccache_dir, pkg_version, pkg_isc_version,
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
repository_url, pkgs_dir):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Prepare local system for Kea development based on requested features.
If tarball_path is provided then instead of Kea sources from current directory
use provided tarball.
"""
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
env = os.environ.copy()
env['LANGUAGE'] = env['LANG'] = env['LC_ALL'] = 'C'
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
system, revision = get_system_revision()
hammer: added support for debian-10, some other various fixes
2019-06-24 21:58:48 +02:00
log.info('Building for %s %s', system, revision)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
execute('sudo df -h', dry_run=dry_run)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
# NOTE: the old code expects having tarball in tarball_path
if not tarball_path:
name_ver = 'kea-%s' % pkg_version
cmd = 'tar --transform "flags=r;s|^|%s/|" --exclude hammer ' % name_ver
cmd += ' --exclude "*~" --exclude .git --exclude .libs '
cmd += ' --exclude .deps --exclude \'*.o\' --exclude \'*.lo\' '
cmd += ' -zcf /tmp/%s.tar.gz .' % name_ver
execute(cmd)
tarball_path = '/tmp/%s.tar.gz' % name_ver
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if 'native-pkg' in features:
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
_build_native_pkg(system, revision, features, tarball_path, env, check_times, dry_run, ccache_dir,
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
pkg_version, pkg_isc_version, repository_url, pkgs_dir)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
else:
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
_build_binaries_and_run_ut(system, revision, features, tarball_path, env, check_times, jobs,
dry_run, ccache_dir)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
execute('sudo df -h', dry_run=dry_run)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
def build_in_vagrant(provider, system, revision, features, leave_system, tarball_path,
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
dry_run, quiet, clean_start, check_times, jobs, ccache_dir,
pkg_version, pkg_isc_version, upload, repository_url):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Build Kea via Vagrant in specified system with specified features."""
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
log.info('')
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
log.info(">>> Building %s, %s, %s", provider, system, revision)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
log.info('')
t0 = time.time()
hammer: fixed reference to undefined ve variable
2019-01-22 15:48:34 +01:00
ve = None
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
error = None
total = 0
passed = 0
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
try:
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
ve = VagrantEnv(provider, system, revision, features, 'kea', dry_run, quiet, check_times,
ccache_dir)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if clean_start:
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
ve.destroy()
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
ve.init_files()
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
ve.bring_up_latest_box()
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
ve.prepare_system()
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
total, passed = ve.run_build_and_test(tarball_path, jobs, pkg_version, pkg_isc_version, upload, repository_url)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
msg = ' - ' + green('all ok')
except KeyboardInterrupt as e:
error = e
msg = ' - keyboard interrupt'
except ExecutionError as e:
error = e
msg = ' - ' + red(str(e))
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
except Exception as e:
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
log.exception('Building erred')
error = e
msg = ' - ' + red(str(e))
finally:
hammer: fixed reference to undefined ve variable
2019-01-22 15:48:34 +01:00
if not leave_system and ve:
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
ve.destroy()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
t1 = time.time()
dt = int(t1 - t0)
log.info('')
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
log.info(">>> Building %s, %s, %s completed in %s:%s%s", provider, system, revision, dt // 60, dt % 60, msg)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
log.info('')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
return dt, error, total, passed
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
def package_box(provider, system, revision, features, dry_run, check_times, reuse, skip_upload):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Prepare Vagrant box of specified system."""
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
ve = VagrantEnv(provider, system, revision, features, 'bare', dry_run, check_times=check_times)
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
if not reuse:
ve.destroy()
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
ve.init_files()
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
ve.bring_up_latest_box()
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
ve.prepare_system()
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
ve.prepare_for_boxing()
hammer improvements - improved updating box when there is newer available - improved pkgs installation by skipping installing when pkgs are already present - added uploading prepared base kea box to vagrant cloud - added measuring time of preparation - completed installing and configuring posgresql on debian and old ubuntu - fixed installing cassandra on centos 7 - added aggregating unit test results
2019-03-25 09:02:18 +01:00
box_path = ve.package()
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
if not skip_upload:
ve.upload_to_cloud(box_path)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
def ssh(provider, system, revision):
"""Invoke Vagrant ssh for given system."""
ve = VagrantEnv(provider, system, revision, [], 'kea', False)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
ve.up()
ve.ssh()
[#2111] hammer: attempt to upgrade vagrant when the latest vagrant version is unknown
2021-11-03 19:50:49 +02:00
def _install_vagrant(ver=RECOMMENDED_VAGRANT_VERSION, upgrade=False):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
system, _ = get_system_revision()
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
if system in ['fedora', 'centos', 'rhel', 'rocky']:
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
if upgrade:
[#2111] hammer: attempt to upgrade vagrant when the latest vagrant version is unknown
2021-11-03 19:50:49 +02:00
execute('sudo yum remove -y vagrant')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('mkdir -p ~/.hammer-tmp')
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
rpm = 'vagrant_%s_x86_64.rpm' % ver
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
cmd = 'wget --no-verbose -O ~/.hammer-tmp/%s ' % rpm
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
cmd += 'https://releases.hashicorp.com/vagrant/%s/%s' % (ver, rpm)
execute(cmd)
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('sudo rpm -i ~/.hammer-tmp/%s' % rpm)
execute('rm -rf ~/.hammer-tmp')
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
elif system in ['debian', 'ubuntu']:
if upgrade:
[#2111] hammer: attempt to upgrade vagrant when the latest vagrant version is unknown
2021-11-03 19:50:49 +02:00
execute('sudo apt-get purge -y vagrant')
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('mkdir -p ~/.hammer-tmp')
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
deb = 'vagrant_%s_x86_64.deb' % ver
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
cmd = 'wget --no-verbose -O ~/.hammer-tmp/%s ' % deb
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
cmd += 'https://releases.hashicorp.com/vagrant/%s/%s' % (ver, deb)
execute(cmd)
[#2657] hammer.py: /tmp -> ~/.hammer-tmp
2022-11-29 20:59:19 +02:00
execute('sudo dpkg -i ~/.hammer-tmp/%s' % deb)
execute('rm -rf ~/.hammer-tmp')
[#2112] hammer.py: support for Arch Linux
2021-11-03 20:08:46 +02:00
elif system in ['arch']:
pass
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
else:
# TODO: check for packages here: https://www.vagrantup.com/downloads.html
[hammer] added support for building kea on alpine and producing apk packages
2019-09-26 15:20:05 +02:00
raise NotImplementedError('no implementation for %s' % system)
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
def ensure_hammer_deps():
"""Install Hammer dependencies onto current, host system."""
exitcode, out = execute('vagrant version', raise_error=False, capture=True)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if exitcode != 0:
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
_install_vagrant()
else:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
m = re.search(r'Installed Version: ([\d\.]+)', out, re.I)
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
ver = m.group(1)
[#2111] hammer: attempt to upgrade vagrant when the latest vagrant version is unknown
2021-11-03 19:50:49 +02:00
vagrant = [int(v) for v in ver.split('.')]
recommended_vagrant = [int(v) for v in RECOMMENDED_VAGRANT_VERSION.split('.')]
if vagrant < recommended_vagrant:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
m = re.search(r'Latest Version: ([\d\.]+)', out, re.I)
[#2111] hammer: attempt to upgrade vagrant when the latest vagrant version is unknown
2021-11-03 19:50:49 +02:00
if m is None:
# Vagrant was unable to check for the latest version of Vagrant.
# Attempt to upgrade to the recommended version to fix it.
_install_vagrant(upgrade=True)
return
hammer: added checking vagrant version, removed hammer from EXTRA_DIST
2019-03-04 11:36:57 +01:00
ver = m.group(1)
_install_vagrant(ver, upgrade=True)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
exitcode = execute('vagrant plugin list | grep vagrant-lxc', raise_error=False)
if exitcode != 0:
execute('vagrant plugin install vagrant-lxc')
[#2064] hammer.py: install lxc on Debian (extra)
2021-09-06 20:32:08 +03:00
# Install lxc-create.
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
system, _ = get_system_revision()
[#2064] hammer.py: install lxc on Debian (extra)
2021-09-06 20:32:08 +03:00
if system == 'debian':
execute('sudo apt-get -y install lxc')
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
if system in ['ubuntu']:
execute('sudo apt-get -y install lxc-utils')
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
class CollectCommaSeparatedArgsAction(argparse.Action):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
"""Helper argparse action class that can split multi-argument options by space and by comma."""
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
def __call__(self, parser, namespace, values, option_string=None):
values2 = []
for v1 in values:
for v2 in v1.split():
values2.extend(v2.split(','))
for v in values2:
if v not in ALL_FEATURES:
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
msg = "feature '%s' is not supported. List of supported features: %s."
msg = msg % (v, ", ".join(ALL_FEATURES))
raise argparse.ArgumentError(self, msg)
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
setattr(namespace, self.dest, values2)
[#2353] hammer: add 'tls' feature to configure TLS in MySQL
2022-03-11 16:38:50 +02:00
DEFAULT_FEATURES = ['docs', 'install', 'perfdhcp', 'unittest']
ALL_FEATURES = ['all', 'ccache', 'distcheck', 'docs', 'forge', 'gssapi',
'install', 'mysql', 'native-pkg', 'netconf', 'perfdhcp',
[#3128] freeradius removed from hammer.py
2023-12-01 13:43:07 +01:00
'pgsql', 'shell', 'tls', 'unittest']
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def parse_args():
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Parse arguments."""
# used lambda to change args order and able to substitute width
fl = functools.partial(lambda w, t: textwrap.fill(t, w), 80)
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
description = [
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
"Hammer - Kea development environment management tool.\n",
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
fl("At first it is required to install Hammer dependencies which is Vagrant and either "
"VirtualBox or LXC. To make life easier Hammer can install Vagrant and required "
"Vagrant plugins using the command:"),
"\n ./hammer.py ensure-hammer-deps\n",
"Still VirtualBox and LXC need to be installed manually.",
fl("Basic functionality provided by Hammer is preparing building environment and "
"performing actual build and running unit tests locally, in current system. "
"This can be achieved by running the command:"),
"\n ./hammer.py build -p local\n",
fl("The scope of the process can be defined using --with (-w) and --without (-x) options. "
"By default the build command will build Kea with documentation, install it locally "
"and run unit tests."),
"To exclude installation and generating docs do:",
"\n ./hammer.py build -p local -x install docs\n",
fl("The whole list of available features is: %s." % ", ".join(ALL_FEATURES)),
fl("Hammer can be told to set up a new virtual machine with specified operating system "
"and not running the build:"),
"\n ./hammer.py prepare-system -p virtualbox -s freebsd -r 12.0\n",
fl("This way we can prepare a system for our own use. To get to such system using SSH invoke:"),
"\n ./hammer.py ssh -p virtualbox -s freebsd -r 12.0\n",
"To list all created system on a host invoke:",
"\n ./hammer.py created-systems\n",
"And then to destroy a given system run:",
"\n ./hammer.py destroy -d /path/to/dir/with/Vagrantfile\n",
]
description = "\n".join(description)
main_parser = argparse.ArgumentParser(description=description,
formatter_class=argparse.RawDescriptionHelpFormatter)
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
main_parser.add_argument('-v', '--verbose', action='store_true', help='Enable verbose mode.')
main_parser.add_argument('-q', '--quiet', action='store_true', help='Enable quiet mode.')
subparsers = main_parser.add_subparsers(dest='command',
title="Hammer commands",
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
description=fl("The following commands are provided by Hammer. "
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"To get more information about particular command invoke: "
"./hammer.py <command> -h."))
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parent_parser1 = argparse.ArgumentParser(add_help=False)
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
parent_parser1.add_argument('-p', '--provider', default='virtualbox',
choices=['lxc', 'virtualbox', 'local', 'all'],
help="Backend build executor. If 'all' then build is executed several times "
"on all providers. If 'local' then build is executed on current system. "
"Default is 'virtualbox'.")
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parent_parser1.add_argument('-s', '--system', default='all', choices=list(SYSTEMS.keys()) + ['all'],
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
help="Build is executed on selected system. If 'all' then build is executed "
"several times on all systems. If provider is 'local' then this option is ignored. "
"Default is 'all'.")
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parent_parser1.add_argument('-r', '--revision', default='all',
help="Revision of selected system. If 'all' then build is executed several times "
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"on all revisions of selected system. To list supported systems and their revisions "
"invoke 'supported-systems'. Default is 'all'.")
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parent_parser2 = argparse.ArgumentParser(add_help=False)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
hlp = "Enable features. Separate them by space or comma. List of available features: %s. Default is '%s'."
hlp = hlp % (", ".join(ALL_FEATURES), ' '.join(DEFAULT_FEATURES))
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
parent_parser2.add_argument('-w', '--with', metavar='FEATURE', nargs='+', default=set(),
action=CollectCommaSeparatedArgsAction, help=hlp)
hlp = "Disable features. Separate them by space or comma. List of available features: %s. Default is ''."
hlp = hlp % ", ".join(ALL_FEATURES)
parent_parser2.add_argument('-x', '--without', metavar='FEATURE', nargs='+', default=set(),
action=CollectCommaSeparatedArgsAction, help=hlp)
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
parent_parser2.add_argument('--with-randomly', metavar='FEATURE', nargs='+', default=set(),
action=CollectCommaSeparatedArgsAction, help=hlp)
[#2311] hammer.py now installs libyang2 and sysrepo2
2022-10-07 15:27:08 +03:00
parent_parser2.add_argument('--ignore-errors-for', metavar='FEATURE', nargs='+', default=set(),
action=CollectCommaSeparatedArgsAction, help=hlp)
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parent_parser2.add_argument('-l', '--leave-system', action='store_true',
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
help='At the end of the command do not destroy vagrant system. Default behavior is '
[#1760] spell checks
2021-03-18 10:22:04 +02:00
'destroying the system.')
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
parent_parser2.add_argument('-c', '--clean-start', action='store_true',
help='If there is pre-existing system then it is destroyed first.')
parent_parser2.add_argument('-i', '--check-times', action='store_true',
[#1760] spell checks
2021-03-18 10:22:04 +02:00
help='Do not allow executing commands infinitely.')
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parent_parser2.add_argument('-n', '--dry-run', action='store_true', help='Print only what would be done.')
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
parser = subparsers.add_parser('ensure-hammer-deps',
help="Install Hammer dependencies on current, host system.")
parser = subparsers.add_parser('supported-systems',
help="List system supported by Hammer for doing Kea development.")
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parser = subparsers.add_parser('build', help="Prepare system and run Kea build in indicated system.",
parents=[parent_parser1, parent_parser2])
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
parser.add_argument('-j', '--jobs', default=0,
help='Number of processes used in compilation. Override make -j default value.')
[#372,!181] hammer: improved docs, added hammer.py to EXTRA_DIST
2019-01-28 12:16:33 +01:00
parser.add_argument('-t', '--from-tarball', metavar='TARBALL_PATH',
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
help='Instead of building sources in current folder use provided tarball '
'package (e.g. tar.gz).')
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
parser.add_argument('--ccache-dir', default=None,
help='Path to CCache directory on host system.')
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
parser.add_argument('--pkg-version', default='0.0.1',
help='Kea version.')
parser.add_argument('--pkg-isc-version', default='isc0',
help='ISC build version of Kea.')
parser.add_argument('--upload', action='store_true',
help='Request uploading native packages to repository indicated by --repository-url.')
parser.add_argument('--repository-url', default=None,
help='Repository for 3rd party dependencies and for uploading built packages.')
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parser = subparsers.add_parser('prepare-system',
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
help="Prepare system for doing Kea development i.e. install all required "
"dependencies and pre-configure the system. build command always first calls "
"prepare-system internally.",
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parents=[parent_parser1, parent_parser2])
[#3229] hammer.py prepare-system --just-configure
2024-01-23 14:57:08 +02:00
parser.add_argument('--just-configure', action='store_true',
help='Whether to prevent installation of packages and only proceed to set them up. '
'Only has an effect when preparing system locally, as opposed to inside vagrant.')
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
parser.add_argument('--ccache-dir', default=None,
help='Path to CCache directory on host system.')
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
parser.add_argument('--repository-url', default=None,
help='Repository for 3rd party dependencies and for uploading built packages.')
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parser = subparsers.add_parser('ssh', help="SSH to indicated system.",
formatter_class=argparse.RawDescriptionHelpFormatter,
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
description="Allows getting into the system using SSH. If the system is "
"not present then it will be created first but not prepared. The command "
"can be run in 2 way: \n"
"1) ./hammer.py ssh -p <provider> -s <system> -r <revision>\n"
"2) ./hammer.py ssh -d <path-to-vagrant-dir>",
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parents=[parent_parser1])
parser.add_argument('-d', '--directory', help='Path to directory with Vagrantfile.')
parser = subparsers.add_parser('created-systems', help="List ALL systems created by Hammer.")
parser = subparsers.add_parser('destroy', help="Destroy indicated system.",
description="Destroys system indicated by a path to directory with Vagrantfile. "
"To get the list of created systems run: ./hammer.py created-systems.")
parser.add_argument('-d', '--directory', help='Path to directory with Vagrantfile.')
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
parser = subparsers.add_parser('package-box',
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
help="Prepare system from scratch and package it into Vagrant Box. "
"Prepared box can be later deployed to Vagrant Cloud.",
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
parents=[parent_parser1, parent_parser2])
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
parser.add_argument('--repository-url', default=None,
help='Repository for 3rd party dependencies and for uploading built packages.')
hammer: added support for fedora 30 and more - added support for fedora 30 - improved creating system boxes - fixed installing cassandra and postgres dependencies on fedora 30
2019-05-14 08:11:34 +02:00
parser.add_argument('-u', '--reuse', action='store_true',
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
help='Reuse existing system image, otherwise (default case) if there is any existing then '
'destroy it first.')
[#1752] hammer: fixed creating system for HA testing in Forge
2021-03-15 09:54:46 +01:00
parser.add_argument('-k', '--skip-upload', action='store_true',
help='Skip uploading prepared box to cloud, otherwise (default case) upload it.')
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
args = main_parser.parse_args()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: added printing help when no command is provided
2019-01-31 14:51:58 +01:00
return args, main_parser
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
def list_supported_systems():
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""List systems hammer can support (with supported providers)."""
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
for system, revision in SYSTEMS.items():
print(f'{system}:')
for release, supported in revision.items():
[#2969] update supported-systems list in hammer.py
2023-10-10 18:51:23 +03:00
if not supported:
continue
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
providers = []
for p in ['lxc', 'virtualbox']:
[#3249] hammer fix - hammer supported-systems works again
2024-02-09 13:46:54 +01:00
k = '%s-%s-%s' % (system, release, p)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if k in IMAGE_TEMPLATES:
providers.append(p)
providers = ', '.join(providers)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
print(f' - {release}: {providers}')
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
def list_created_systems():
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""List VMs that are created on this host by Hammer."""
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
_, output = execute('vagrant global-status --prune', quiet=True, capture=True)
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
systems = []
for line in output.splitlines():
if 'hammer' not in line:
continue
elems = line.split()
state = elems[3]
path = elems[4]
systems.append([path, state])
print('')
print('%-10s %s' % ('State', 'Path'))
print('-' * 80)
for path, state, in sorted(systems):
print('%-10s %s' % (state, path))
print('-' * 80)
print('To destroy a system run: ./hammer.py destroy -d <path>')
print('')
def destroy_system(path):
[#1426] cleanup in release checklist template and in hammer
2020-09-23 13:45:18 +02:00
"""Destroy Vagrant system under given path."""
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
vf = os.path.join(path, 'Vagrantfile')
if not os.path.exists(vf):
print('Wrong directory. It does not have Vagrantfile.')
sys.exit(1)
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
execute('vagrant destroy', cwd=path, interactive=True)
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
def _coin_toss():
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
# Issue: [B311:blacklist] Standard pseudo-random generators are not suitable for security/cryptographic
# purposes.
# Reason for nosec: It is not used in a security context.
if random.randint(0, 65535) % 2 == 0: # nosec B311
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
return True
return False
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
def _get_features(args):
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
features = set(vars(args)['with'])
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
# establish initial set of features
if 'all' in features:
[#2539] hammer.py: never discard features or prevent their installation At most, incompatible features should log an error and make hammer exit. But there are no incompatible features to my knowledge.
2022-09-05 22:28:01 +03:00
# special case 'all' but some features need to be removed
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
# as they are not compatible with others
features = set(ALL_FEATURES)
features.discard('all')
[#2934] fix ignoring of native-pkg parameter
2023-07-18 11:56:44 +00:00
# do not include `native-pkg` in `all` if not set explicitly in parameters
[#2934] fix ignoring of native-pkg parameter
2023-07-06 08:04:07 +00:00
if 'native-pkg' not in set(vars(args)['with']):
features.discard('native-pkg')
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
# if we build native packages then some features are required and some not
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
if 'native-pkg' in features:
features.add('docs')
features.add('perfdhcp')
features.add('shell')
features.add('mysql')
features.add('pgsql')
[#1885] hammer: added support for building kea with gss-api
2021-06-28 12:19:36 +02:00
features.add('gssapi')
hammer: added support for building kea on alpine 3.13, ubuntu 21.04, fedora 34, freebsd 13
2021-06-07 14:52:51 +02:00
# in case of build command of native packages, unittest should not
# be run as they are not built
if args.command == 'build':
features.discard('unittest')
[#1072] support building kea on fedora 31 and ubuntu 19.10 - added this support in hammer.py - fixed checking sphinx in configure.ac on fedora 31
2019-12-20 09:58:04 +01:00
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
nofeatures = set(args.without)
features = features.difference(nofeatures)
for i in args.with_randomly:
if _coin_toss():
features.add(i)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('Feature enabled through coin toss: %s', i)
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
else:
features.discard(i)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.info('Feature disabled through coin toss: %s', i)
[#2539] add --with-randomly to hammer.py For testing both with and without a feature over time, you might want to enable or disable it non-determinstically for a certain test run. This can be done with the --with-randomly parameter which has a 50% chance of enabling a feature and a 50% chance of disabling it.
2022-09-05 19:19:09 +03:00
if hasattr(args, 'ccache_dir') and args.ccache_dir:
features.add('ccache')
[#2539] hammer.py: revert sorting of features Fixes TypeError: unsupported operand type(s) for -: 'set' and 'list'
2022-10-03 11:16:38 +03:00
return features
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
def _print_summary(results, features):
[#1760] spell checks
2021-03-18 10:22:04 +02:00
"""Print summary of build times and unit-test results."""
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
print("")
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
print("+===== Hammer Summary ====================================================+")
print("| provider | system | revision | duration | status | unit tests |")
print("+------------+------------+----------+-----------+---------+--------------+")
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
total_dt = 0
for key, result in results.items():
provider, system, revision = key
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
dt, error, ut_total, ut_passed = result
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
total_dt += dt
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
if error is None:
status = ' %s' % green('ok')
elif error == 'not run':
status = blue('not run')
else:
status = ' %s' % red('error')
if 'unittest' in features:
ut_results = '%s/%s' % (ut_passed, ut_total)
padding = ' ' * (12 - len(ut_results))
if ut_passed < ut_total or ut_total == 0:
ut_results = padding + red(ut_results)
else:
ut_results = padding + green(ut_results)
else:
ut_results = ' not planned'
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
txt = '| %10s | %10s | %8s | %6d:%02d | %s | %s |' % (provider, system, revision,
dt // 60, dt % 60, status, ut_results)
print(txt)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
print("+------------+------------+----------+-----------+---------+--------------+")
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
txt = "| Total: %6d:%02d | |" % (total_dt // 60,
total_dt % 60)
print(txt)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
print("+=========================================================================+")
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: added checking system revision, fixed prepare-system
2019-01-28 17:32:26 +01:00
def _check_system_revision(system, revision):
[#372,!181] hammer: lots of improvements: - added comments and documentation, - added missing passing env to execute calls, - fixed checking revision, - removed unnecessary code
2019-01-29 11:30:05 +01:00
if revision == 'all':
return
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
if system not in SYSTEMS:
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
msg = "hammer.py error: argument -s/--system: invalid choice: '%s' (choose from '%s')"
msg = msg % (revision, "', '".join(SYSTEMS.keys()))
log.error(msg)
sys.exit(1)
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
if revision not in SYSTEMS[system]:
[#372,!181] hammer: added checking system revision, fixed prepare-system
2019-01-28 17:32:26 +01:00
msg = "hammer.py error: argument -r/--revision: invalid choice: '%s' (choose from '%s')"
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
msg = msg % (revision, "', '".join(SYSTEMS[system].keys()))
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
log.error(msg)
[#372,!181] hammer: added checking system revision, fixed prepare-system
2019-01-28 17:32:26 +01:00
sys.exit(1)
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
if not SYSTEMS[system][revision]:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
log.warning('%s %s is no longer officially supported. '
'The script will continue in a best-effort manner.', system, revision)
[#372,!181] hammer: added checking system revision, fixed prepare-system
2019-01-28 17:32:26 +01:00
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
def _prepare_ccache_dir(ccache_dir, system, revision):
if not ccache_dir:
return None
ccache_dir = os.path.join(ccache_dir, "%s-%s" % (system, revision))
ccache_dir = os.path.abspath(ccache_dir)
if not os.path.exists(ccache_dir):
os.makedirs(ccache_dir)
return ccache_dir
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
def prepare_system_cmd(args):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Check command args and run the prepare-system command."""
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if args.provider != 'local' and (args.system == 'all' or args.revision == 'all'):
print('Please provide required system and its version.')
print('Example: ./hammer.py prepare-system -s fedora -r 28.')
print('To get list of supported systems run: ./hammer.py supported-systems.')
sys.exit(1)
_check_system_revision(args.system, args.revision)
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
features = _get_features(args)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
log.info('Enabled features: %s', ' '.join(features))
if args.provider == 'local':
[#3229] hammer.py prepare-system --just-configure
2024-01-23 14:57:08 +02:00
prepare_system_local(features, args.check_times, args.ignore_errors_for, args.just_configure)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
return
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
ccache_dir = _prepare_ccache_dir(args.ccache_dir, args.system, args.revision)
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
_check_deps_presence()
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
prepare_system_in_vagrant(args.provider, args.system, args.revision, features,
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
args.dry_run, args.check_times, args.clean_start,
ccache_dir)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
def upload_to_repo(args, pkgs_dir):
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
# NOTE: note the differences (if any) in system/revision vs args.system/revision
system, revision = get_system_revision()
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
repo_url = _get_full_repo_url(args.repository_url, system, revision)
[#3287] fix bandit warnings
2024-05-07 13:28:48 +03:00
if repo_url is None:
raise ValueError('repo_url is None')
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
upload_cmd = 'curl -v --netrc -f'
log.info('args.system %s, system = %s', args.system, system)
file_ext = ''
if system in ['ubuntu', 'debian']:
upload_cmd += ' -X POST -H "Content-Type: multipart/form-data" --data-binary "@%s" '
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
file_ext = 'deb' # include both '.deb' and '.ddeb' files
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
[#3247] add rocky linux 9
2024-04-24 11:29:29 +00:00
elif system in ['fedora', 'centos', 'rhel', 'rocky']:
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
upload_cmd += ' --upload-file %s '
file_ext = '.rpm'
elif system == 'alpine':
upload_cmd += ' --upload-file %s '
file_ext = ''
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
_, arch = execute('arch', raise_error=False, capture=True)
arch = arch.strip()
repo_url = urljoin(repo_url, f'{args.pkg_isc_version}/v{revision}/{arch}/')
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
upload_cmd += ' ' + repo_url
for fn in os.listdir(pkgs_dir):
log.info("debug: fn = %s", fn)
if file_ext and not fn.endswith(file_ext):
log.info('File extension "%s" is not supported by upload_to_repo function', file_ext)
continue
fp = os.path.join(pkgs_dir, fn)
log.info("upload cmd: %s", upload_cmd)
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
log.info("file path: %s", fp)
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
cmd = upload_cmd % fp
[#2058] hammer.py: retry upload on 504
2021-09-02 15:02:08 +03:00
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
attempts = 4
[#2058] hammer.py: retry upload on 504
2021-09-02 15:02:08 +03:00
while attempts > 0:
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
exitcode, output = execute(cmd, capture=True, raise_error=False)
[#2058] hammer.py: retry upload on 504
2021-09-02 15:02:08 +03:00
if exitcode != 0 and '504 Gateway Time-out' in output:
log.info('Trying again after 8 seconds...')
attempts -= 1
time.sleep(8)
[#3186] arm packages can be build and uploaded, new systems support added
2024-01-03 10:49:14 +00:00
elif exitcode != 0 and "pository does not allow updating assets" in output:
log.info("Asset already exists in the repository. Skipping upload.")
break
elif exitcode != 0:
[#3287] fix pylint warnings - C0115: Missing class docstring (missing-class-docstring) - C0123: Use isinstance() rather than type() for a typecheck. (unidiomatic-typecheck) - C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) - C0206: Consider iterating with .items() (consider-using-dict-items) - C0411: standard import "..." should be placed before "..." (wrong-import-order) - C0415: Import outside toplevel (...) (import-outside-toplevel) - C1802: Do not use `len(SEQUENCE)` without comparison to determine if a sequence is empty (use-implicit-booleaness-not-len) - E0001: Parsing failed: 'invalid syntax (<unknown>, line 2313)' (syntax-error) - E0401: Unable to import '...' (import-error) - E0602: Undefined variable 'l' (undefined-variable) - R0205: Class 'VagrantEnv' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance) - E1101: Instance of 'NSECBASE' has no 'dump_fixedpart' member (no-member) - E1123: Unexpected keyword argument 'capture' in method call (unexpected-keyword-arg) - R0902: Too many instance attributes (too-many-instance-attributes) - R0913: Too many arguments (too-many-arguments) - R0916: Too many boolean expressions in if statement (6/5) (too-many-boolean-expressions) - R1717: Consider using a dictionary comprehension (consider-using-dict-comprehension) - R1722: Consider using 'sys.exit' instead (consider-using-sys-exit) - R1732: Consider using 'with' for resource-allocating operations (consider-using-with) - R1735: Consider using '{}' instead of a call to 'dict'. (use-dict-literal) - W0102: Dangerous default value sys.argv[1:] (builtins.list) as argument (dangerous-default-value) - W0102: Dangerous default value {} as argument (dangerous-default-value) - W0106: Expression "[f.write('%02x' % x) for x in bin_address]" is assigned to nothing (expression-not-assigned) - W0107: Unnecessary pass statement (unnecessary-pass) - W0201: Attribute 'config' defined outside __init__ (attribute-defined-outside-init) - W0404: Reimport '...' (imported line ...) (reimported) - W0611: Unused import ... (unused-import) - W0612: Unused variable '...' (unused-variable) - W0613: Unused argument '...' (unused-argument) - W0621: Redefining name '...' from outer scope (line 1471) (redefined-outer-name) - W0622: Redefining built-in '...' (redefined-builtin) - W0707: Consider explicitly re-raising using 'raise ... from ...' (raise-missing-from) - W0718: Catching too general exception Exception (broad-exception-caught) - W1202: Use lazy % formatting in logging functions (logging-format-interpolation) - W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation) - W1308: Duplicate string formatting argument 'connection_type', consider passing as named argument (duplicate-string-formatting-argument) - W1401: Anomalous backslash in string: '\/'. String constant might be missing an r prefix. (anomalous-backslash-in-string) - W1406: The u prefix for strings is no longer necessary in Python >=3.0 (redundant-u-string-prefix) - W1514: Using open without explicitly specifying an encoding (unspecified-encoding) - W4901: Deprecated module 'optparse' (deprecated-module) - W4904: Using deprecated class SafeConfigParser of module configparser (deprecated-class)
2024-06-07 11:43:43 +03:00
raise UnexpectedError('Upload failed: %s' % output)
[#2058] hammer.py: retry upload on 504
2021-09-02 15:02:08 +03:00
else:
break
[#2058] hammer.py: indentation (minor change)
2021-08-27 18:54:59 +03:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
def build_cmd(args):
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Check command args and run the build command."""
features = _get_features(args)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
log.info('Enabled features: %s', ' '.join(features))
if args.provider == 'local':
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
# NOTE: working dir is /tmp/workspace/kea-dev/pkg
pkgs_dir = "kea-pkg"
if os.path.exists(pkgs_dir):
execute('rm -rf %s' % pkgs_dir)
os.makedirs(pkgs_dir)
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
build_local(features, args.from_tarball, args.check_times, int(args.jobs), args.dry_run,
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
args.ccache_dir, args.pkg_version, args.pkg_isc_version, args.repository_url, pkgs_dir)
# NOTE: upload the locally build packages and leave; the rest of the code is vagrant specific
if args.upload:
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
upload_to_repo(args, pkgs_dir)
[#1761] hammer: Jenkins migration to AWS
2021-02-16 11:50:25 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
return
_check_system_revision(args.system, args.revision)
added check if --repository-url is passed when native-pkg is enabled
2019-05-15 17:15:43 +02:00
if 'native-pkg' in features and not args.repository_url:
msg = "Enabling 'native-pkg' requires passing --repository-url."
print(msg)
sys.exit(1)
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
_check_deps_presence()
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
if args.provider == 'all':
providers = ['lxc', 'virtualbox']
else:
providers = [args.provider]
if args.system == 'all':
systems = SYSTEMS.keys()
else:
systems = [args.system]
plan = []
results = {}
log.info('Build plan:')
for provider in providers:
for system in systems:
if args.revision == 'all':
[#2969] fix prepare-system in hammer.py - fixes this error: ``` $ ./hammer.py prepare-system -p local -s fedora -r 36 -w all File "hammer.py", line 3031, in _check_system_revision msg = msg % (revision, "', '".join(revs)) TypeError: sequence item 0: expected str instance, tuple found ``` - also marks CentOS, FreeBSD 12, and Alpine 3.15 as unsupported
2023-11-06 12:59:08 +02:00
revisions = SYSTEMS[system].keys()
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
else:
revisions = [args.revision]
for revision in revisions:
if args.revision == 'all':
key = '%s-%s-%s' % (system, revision, provider)
if key not in IMAGE_TEMPLATES:
continue
plan.append((provider, system, revision))
log.info(' - %s, %s, %s', provider, system, revision)
results[(provider, system, revision)] = (0, 'not run')
fail = False
for provider, system, revision in plan:
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
ccache_dir = _prepare_ccache_dir(args.ccache_dir, args.system, args.revision)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
result = build_in_vagrant(provider, system, revision, features, args.leave_system, args.from_tarball,
added CCache support to Hammer
2019-03-14 16:37:12 +01:00
args.dry_run, args.quiet, args.clean_start, args.check_times, int(args.jobs),
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
ccache_dir, args.pkg_version, args.pkg_isc_version, args.upload, args.repository_url)
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
results[(provider, system, revision)] = result
error = result[1]
if error:
fail = True
if isinstance(error, KeyboardInterrupt):
break
_print_summary(results, features)
if fail:
sys.exit(1)
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
def _check_deps_presence():
[#1512] hammer: improved handling missing vagrant
2021-01-05 15:37:38 +01:00
ret = execute('vagrant -v', super_quiet=True, raise_error=False)
if ret != 0:
print('Missing vagrant. Please install it from https://www.vagrantup.com/')
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
sys.exit(1)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
def main():
[#372,!181] hammer: reduced line lenghts, added docstrings, unified naming, fixed terminating timed out process
2019-01-31 11:36:38 +01:00
"""Main function - parse args and invoke proper command."""
[#372,!181] hammer: added printing help when no command is provided
2019-01-31 14:51:58 +01:00
args, parser = parse_args()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# prepare logging
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if args.verbose:
level = logging.DEBUG
improved support for building native packages: rpm and deb
2019-04-10 10:50:15 +02:00
fmt = '[HAMMER] %(asctime)-15s L%(lineno)04d %(message)s'
else:
level = logging.INFO
fmt = '[HAMMER] %(asctime)-15s %(message)s'
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
logging.basicConfig(format=fmt, level=level)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
# dispatch command
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
if args.command == 'supported-systems':
list_supported_systems()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
elif args.command == 'created-systems':
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
_check_deps_presence()
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
list_created_systems()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif args.command == "package-box":
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
_check_deps_presence()
[#372,!181] hammer: added checking system revision, fixed prepare-system
2019-01-28 17:32:26 +01:00
_check_system_revision(args.system, args.revision)
[#3128] freeradius removed from hammer.py
2023-12-01 13:43:07 +01:00
features = set(['docs', 'perfdhcp', 'shell', 'mysql', 'pgsql', 'gssapi', 'native-pkg'])
[#1154] when lxc box is prepared it should contain all deps for building kea
2020-03-19 12:48:18 +01:00
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
log.info('Enabled features: %s', ' '.join(features))
[#3287] fix pycodestyle warnings - E111 indentation is not a multiple of 4 - E117 over-indented - E125 continuation line with same indent as next logical line - E127 continuation line over-indented for visual indent - E128 continuation line under-indented for visual indent - E129 visually indented line with same indent as next logical line - E131 continuation line unaligned for hanging indent - E201 whitespace after '[' - E201 whitespace after '{' - E202 whitespace before ')' - E202 whitespace before ']' - E202 whitespace before '}' - E203 whitespace before ' - E203 whitespace before ',' - E222 multiple spaces after operator - E225 missing whitespace around operator - E228 missing whitespace around modulo operator - E231 missing whitespace after ',' - E251 unexpected spaces around keyword / parameter equals - E261 at least two spaces before inline comment - E262 inline comment should start with '# ' - E265 block comment should start with '# ' - E301 expected 1 blank line, found 0 - E302 expected 2 blank lines, found 1 - E303 too many blank lines (2) - E305 expected 2 blank lines after class or function definition, found 1 - E306 expected 1 blank line before a nested definition, found 0 - E401 multiple imports on one line - E402 module level import not at top of file - E501 line too long - E502 the backslash is redundant between brackets - E703 statement ends with a semicolon - E713 test for membership should be 'not in' - E722 do not use bare 'except' - E741 ambiguous variable name 'l' - W605 invalid escape sequence '\/'
2024-06-06 15:41:10 +03:00
package_box(args.provider, args.system, args.revision, features, args.dry_run, args.check_times, args.reuse,
args.skip_upload)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif args.command == "prepare-system":
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
prepare_system_cmd(args)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif args.command == "build":
[#372,!181] hammer: fixes after running pylint checker
2019-01-29 12:07:09 +01:00
build_cmd(args)
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
elif args.command == "ssh":
[#372,!181] hammer: added checking system revision, fixed prepare-system
2019-01-28 17:32:26 +01:00
_check_system_revision(args.system, args.revision)
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
if not args.system or not args.revision or args.system == 'all' or args.revision == 'all':
print('System (-s) and revision (-r) parameters are required')
sys.exit(1)
_check_deps_presence()
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
ssh(args.provider, args.system, args.revision)
hammer: added handling much more things (more systems, running unit tests with mysql, pgsql and cql)
2019-01-10 17:36:02 +01:00
elif args.command == "ensure-hammer-deps":
ensure_hammer_deps()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
elif args.command == "destroy":
[#1512] hammer: improved handling error conditions
2021-01-05 10:12:00 +01:00
if not args.directory:
print('Missing directory (-d) parameter')
sys.exit(1)
_check_deps_presence()
hammer: improved help and commands, added new commands for listing and destroying existing systems
2019-01-25 16:08:12 +01:00
destroy_system(args.directory)
[#372,!181] hammer: added printing help when no command is provided
2019-01-31 14:51:58 +01:00
else:
parser.print_help()
initial version of Hammer: a tool for building and unittesting Kea
2018-12-27 15:15:08 +01:00
if __name__ == '__main__':
main()
Reference in New Issue Copy Permalink