diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5772c7566..23691d983 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -22,7 +22,7 @@ jobs: fetch-depth: 0 - name: Build and Test - run: cd build/ && ./assemble_api_jar.sh && ./run_all_tests.sh + run: cd build/ && ./assemble_api.sh && ./run_all_tests.sh shell: bash - name: Codecov diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index b8087c285..5bfce78b9 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -206,60 +206,78 @@ You should now be able to see the zone in the portal at localhost:9001 when logg ### Unit Tests -1. First, start up your Scala build tool: `sbt`. Running *clean* immediately after starting is recommended. -1. (Optionally) Go to the project you want to work on, for example `project api` for the API; `project portal` for the +1. First, start up your Scala build tool: `build/sbt.sh` (or `sbt` if running outside of Docker). +2. (Optionally) Go to the project you want to work on, for example `project api` for the API; `project portal` for the portal. -1. Run _all_ unit tests by just running `test`. -1. Run an individual unit test by running `testOnly *MySpec`. -1. If you are working on a unit test and production code at the same time, use `~` (e.g., `~testOnly *MySpec`) to +3. Run _all_ unit tests by just running `test`. +4. Run a single unit test suite by running `testOnly *MySpec`. +5. Run a single unit by filtering the test name using the `-z` argument `testOnly *MySpec -- -z "some text from test"`. + - [More information on commandline arguments](https://www.scalatest.org/user_guide/using_the_runner) +6. If you are working on a unit test and production code at the same time, use `~` (e.g., `~testOnly *MySpec`) to automatically background compile for you! ### Integration Tests -Integration tests are used to test integration with _real_ dependent services. We use Docker to spin up those backend -services for integration test development. +Integration tests are used to test integration with dependent services. We use Docker to spin up those backend services +for integration test development. -1. Type `dockerComposeUp` to start up dependent background services +1. Type `quickstart/quickstart-vinyldns.sh --reset --deps-only` to start up dependent background services +1. Run sbt (`build/sbt.sh` or `sbt` locally) 1. Go to the target module in sbt, example: `project api` 1. Run all integration tests by typing `it:test`. 1. Run an individual integration test by typing `it:testOnly *MyIntegrationSpec` 1. You can background compile as well if working on a single spec by using `~it:testOnly *MyIntegrationSpec` -1. You must stop (`dockerComposeStop`) and start (`dockerComposeUp`) the dependent services from the root - project (`project root`) before you rerun the tests. +1. You must restart the dependent services (`quickstart/quickstart-vinyldns.sh --reset --deps-only`) before you rerun + the tests. 1. For the mysql module, you may need to wait up to 30 seconds after starting the services before running the tests for setup to complete. #### Running both -You can run all unit and integration tests for the api and portal by running `sbt verify` +You can run all unit and integration tests for the api and portal by running `build/verify.sh` ### Functional Tests When adding new features, you will often need to write new functional tests that black box / regression test the API. -- The API functional tests are written in Python and live under `test/api/functional`. -- The Portal functional tests are written in JavaScript and live under `test/portal/functional`. +- The API functional tests are written in Python and live under `modules/api/src/test/functional`. +- The Portal functional tests are written in JavaScript and live under `modules/portal/test`. #### Running Functional Tests -To run functional tests you can simply execute the following command: +To run functional tests you can simply execute the following commands: ``` +build/func-test-api.sh +build/func-test-portal.sh +``` + +These command will run the API functional tests and portal functional tests respectively. + +##### API Functional Tests + +To run functional tests you can simply execute `build/func-test-api.sh`, but if you'd like finer-grained control, you +can work with the `Makefile` in `test/api/functional`: + +``` +cd test/api/functional make build && make run ``` -During iterative test development, you can use `make run-local` which will mount the current functional tests in the -container, allowing for easier test development. +During iterative test development, you can use `make run-local` which will bind-mount the current functional tests in +the container, allowing for easier test development. Additionally, you can pass `--interactive` to `make run` or `make run-local` to drop to a shell inside the container. From there you can run tests with the `/functional_test/run.sh` command. This allows for finer-grained control over the test execution process as well as easier inspection of logs. -##### API Functional Tests - You can run a specific test by name by running `make run -- -k `. Any arguments after `make run --` will be passed to the test runner [`test/api/functional/run.sh`](test/api/functional/run.sh). +Finally, you can execute `make run-deps-bg` to all of the dependencies for the functional test, but not run the tests. +This is useful if, for example, you want to use an interactive debugger on your local machine, but host all of the +VinylDNS API dependencies in Docker. + #### Setup We use [pytest](https://docs.pytest.org/en/latest/) for python tests. It is helpful that you browse the documentation so @@ -269,15 +287,16 @@ We also use [PyHamcrest](https://pyhamcrest.readthedocs.io/en/release-1.8/) for tests. Please browse that documentation as well so that you are familiar with the different matchers for PyHamcrest. There aren't a lot, so it should be quick. -In the `test/api/functional` directory are a few important files for you to be familiar with: +In the `modules/api/src/test/functional` directory are a few important files for you to be familiar with: -* `vinyl_client.py` - this provides the interface to the VinylDNS API. It handles signing the request for you, as well +* `vinyl_python.py` - this provides the interface to the VinylDNS API. It handles signing the request for you, as well as building and executing the requests, and giving you back valid responses. For all new API endpoints, there should be a corresponding function in the vinyl_client * `utils.py` - provides general use functions that can be used anywhere in your tests. Feel free to contribute new functions here when you see repetition in the code -In the `test/api/functional/tests` directory, we have directories / modules for different areas of the application. +In the `modules/api/src/test/functional/tests` directory, we have directories / modules for different areas of the +application. * `batch` - for managing batch updates * `internal` - for internal endpoints (not intended for public consumption) diff --git a/build/README.md b/build/README.md index e1d77c9d3..cb060269e 100644 --- a/build/README.md +++ b/build/README.md @@ -2,12 +2,13 @@ This folder contains scripts for building VinylDNS and it's related artifacts. -| Path |Description | -| --- | --- | -| `assemble_api_jar.sh` | Builds the VinylDNS API jar file. You can find the resulting `jar` file in `assembly/`.| -| `deep_clean.sh` | Removes all of the build artifacts and all `target/` directories recursively.| -| `func-test-api.sh` | Runs the functional tests for the API| -| `func-test-portal.sh` | Runs the functional tests for the Portal| -| `publish_docs.sh` | Publishes the documentation site| -| `run_all_tests.sh` | Runs all of the tests: unit, integration, and functional| -| `verify.sh` | Runs all of the unit and integration tests| +| Path | Description | +|-----------------------|-----------------------------------------------------------------------------------------| +| `assemble_api_jar.sh` | Builds the VinylDNS API jar file. You can find the resulting `jar` file in `assembly/`. | +| `deep_clean.sh` | Removes all of the build artifacts and all `target/` directories recursively. | +| `func-test-api.sh` | Runs the functional tests for the API | +| `func-test-portal.sh` | Runs the functional tests for the Portal | +| `publish_docs.sh` | Publishes the documentation site | +| `run_all_tests.sh` | Runs all of the tests: unit, integration, and functional | +| `sbt.sh` | Runs `sbt` in a Docker container with the current project bind-mounted in `/build` | +| `verify.sh` | Runs all of the unit and integration tests | diff --git a/build/assemble_api.sh b/build/assemble_api.sh index ab9970ce6..a4f18017f 100644 --- a/build/assemble_api.sh +++ b/build/assemble_api.sh @@ -12,16 +12,16 @@ DIR=$( usage() { echo "USAGE: assemble_api.sh [options]" - echo -e "\t-n, --no-clean do no perform a clean before assembling the jar" + echo -e "\t-n, --no-cache do not use cache when building the artifact" echo -e "\t-u, --update update the underlying docker image" } -SKIP_CLEAN=0 +NO_CACHE=0 UPDATE_DOCKER=0 while [[ $# -gt 0 ]]; do case "$1" in - --no-clean | -n) - SKIP_CLEAN=1 + --no-cache | -n) + NO_CACHE=1 shift ;; --update | -u) @@ -35,15 +35,15 @@ while [[ $# -gt 0 ]]; do esac done -if ! [[ $SKIP_CLEAN -eq 1 ]]; then - "${DIR}/deep_clean.sh" - rm "${DIR}/../artifacts/vinyldns-api.jar" &> /dev/null || true +if [[ $NO_CACHE -eq 1 ]]; then + rm -rf "${DIR}/../artifacts/vinyldns-api.jar" &> /dev/null || true + docker rmi vinyldns:api-artifact &> /dev/null || true fi if [[ $UPDATE_DOCKER -eq 1 ]]; then - echo "Pulling latest version of 'vinyldns/build:base-test-integration'" - docker pull vinyldns/build:base-test-integration + echo "Pulling latest version of 'vinyldns/build:base-build'" + docker pull vinyldns/build:base-build fi echo "Building VinylDNS API artifact" -docker run -i --rm -e RUN_SERVICES=none -v "${DIR}/..:/build" vinyldns/build:base-test-integration -- sbt 'api/assembly' +make -C "${DIR}/docker/api" artifact diff --git a/build/assemble_portal.sh b/build/assemble_portal.sh index ad10103c0..ccc6f9ac7 100644 --- a/build/assemble_portal.sh +++ b/build/assemble_portal.sh @@ -12,16 +12,16 @@ DIR=$( usage() { echo "USAGE: assemble_portal.sh [options]" - echo -e "\t-n, --no-clean do no perform a clean before assembling the jar" + echo -e "\t-n, --no-cache do not use cache when building the artifact" echo -e "\t-u, --update update the underlying docker image" } -SKIP_CLEAN=0 +NO_CACHE=0 UPDATE_DOCKER=0 while [[ $# -gt 0 ]]; do case "$1" in --no-clean | -n) - SKIP_CLEAN=1 + NO_CACHE=1 shift ;; --update | -u) @@ -35,16 +35,15 @@ while [[ $# -gt 0 ]]; do esac done -if ! [[ $SKIP_CLEAN -eq 1 ]]; then - "${DIR}/deep_clean.sh" - rm "${DIR}/../artifacts/vinyldns-portal.zip" &> /dev/null || true - rm -rf "${DIR}/../artifacts/scripts" &> /dev/null || true +if [[ $NO_CACHE -eq 1 ]]; then + rm -rf "${DIR}/../artifacts/vinyldns-portal.zip" &> /dev/null || true + docker rmi vinyldns:portal-artifact &> /dev/null || true fi if [[ $UPDATE_DOCKER -eq 1 ]]; then - echo "Pulling latest version of 'vinyldns/build:base-test-integration'" - docker pull vinyldns/build:base-test-integration + echo "Pulling latest version of 'vinyldns/build:base-build'" + docker pull vinyldns/build:base-build fi echo "Building VinylDNS Portal artifact" -docker run -i --rm -e RUN_SERVICES=none -v "${DIR}/..:/build" vinyldns/build:base-test-integration -- sbt 'portal/dist' +make -C "${DIR}/docker/portal" artifact diff --git a/build/deep_clean.sh b/build/deep_clean.sh index b30d950e4..d68681ce9 100755 --- a/build/deep_clean.sh +++ b/build/deep_clean.sh @@ -10,7 +10,6 @@ DIR=$( echo "Performing deep clean" find "${DIR}/.." -type d -name target -o -name assembly | while read -r p; do if [ -d "$p" ]; then - echo -n "Removing $p.." - rm -r "$p" || (echo -e "\e[93mError deleting $p, you may need to be root\e[0m"; exit 1) - echo "done." + echo -n "Removing $(realpath --relative-to="$DIR" "$p").." && \ + { { rm -rf "$p" &> /dev/null && echo "done."; } || { echo -e "\e[93mERROR\e[0m: you may need to be root"; exit 1; } } fi; done diff --git a/build/docker/api/Dockerfile b/build/docker/api/Dockerfile index 89ea8d028..d35082f86 100644 --- a/build/docker/api/Dockerfile +++ b/build/docker/api/Dockerfile @@ -3,13 +3,13 @@ FROM vinyldns/build:base-build as base-build COPY . /build/ WORKDIR /build -## Run the build if we don't already have a vinyldns.jar +## Run the build if we don't already have a vinyldns-api.jar RUN mkdir -p /opt/vinyldns/conf && \ if [ -f artifacts/vinyldns-api.jar ]; then cp artifacts/vinyldns-api.jar /opt/vinyldns/; fi && \ - if [ ! -f /opt/vinyldns/vinyldns.jar ]; then \ + if [ ! -f /opt/vinyldns/vinyldns-api.jar ]; then \ env SBT_OPTS="-XX:+UseConcMarkSweepGC -Xmx4G -Xms1G" \ sbt -Dbuild.scalafmtOnCompile=false -Dbuild.lintOnCompile=fase ";project api;coverageOff;assembly" \ - && cp assembly/vinyldns.jar /opt/vinyldns/; \ + && cp artifacts/vinyldns-api.jar /opt/vinyldns/; \ fi FROM adoptopenjdk/openjdk11:jdk-11.0.8_10-alpine @@ -36,4 +36,4 @@ ENTRYPOINT ["/bin/bash", "-c", "java ${JVM_OPTS} -Dconfig.file=/opt/vinyldns/con -Dlogback.configurationFile=/opt/vinyldns/conf/logback.xml \ -Dvinyldns.version=$(cat /opt/vinyldns/version) \ -cp /opt/vinyldns/lib_extra/* \ - -jar /opt/vinyldns/vinyldns.jar" ] + -jar /opt/vinyldns/vinyldns-api.jar" ] diff --git a/build/docker/api/Makefile b/build/docker/api/Makefile index f9ba5fd59..7f372509c 100644 --- a/build/docker/api/Makefile +++ b/build/docker/api/Makefile @@ -31,6 +31,12 @@ endif all: build run +artifact: + @set -euo pipefail + cd ../../.. + docker build $(BUILD_ARGS) --target base-build --build-arg DOCKER_FILE_PATH="$$(realpath --relative-to="." "$(ROOT_DIR)")" --build-arg VINYLDNS_VERSION="$(IMAGE_TAG)" -t "vinyldns:api-artifact" -f "$(ROOT_DIR)/Dockerfile" . + docker run -it --rm -v "$$(pwd)/:/output" vinyldns:api-artifact /bin/bash -c "cp /build/artifacts/*.jar /output/artifacts" + build: @set -euo pipefail cd ../../.. diff --git a/build/docker/api/application.conf b/build/docker/api/application.conf index 201018765..b1f9eb837 100644 --- a/build/docker/api/application.conf +++ b/build/docker/api/application.conf @@ -168,6 +168,7 @@ vinyldns { secret = ${?CRYPTO_SECRET} } + data-stores = ["mysql"] mysql { settings { # JDBC Settings, these are all values in scalikejdbc-config, not our own @@ -186,6 +187,28 @@ vinyldns { password = "" password = ${?JDBC_PASSWORD} } + + # TODO: Remove the need for these useless configuration blocks + repositories { + zone { + } + batch-change { + } + user { + } + record-set { + } + zone-change { + } + record-change { + } + group { + } + group-change { + } + membership { + } + } } backends = [] diff --git a/build/docker/portal/Makefile b/build/docker/portal/Makefile index be2b8b2a5..ff1ba53cf 100644 --- a/build/docker/portal/Makefile +++ b/build/docker/portal/Makefile @@ -35,6 +35,12 @@ all: build run all: build run +artifact: + @set -euo pipefail + cd ../../.. + docker build $(BUILD_ARGS) --target base-build --build-arg DOCKER_FILE_PATH="$$(realpath --relative-to="." "$(ROOT_DIR)")" --build-arg VINYLDNS_VERSION="$(IMAGE_TAG)" -t "vinyldns:portal-artifact" -f "$(ROOT_DIR)/Dockerfile" . + docker run -it --rm -v "$$(pwd)/:/output" vinyldns:portal-artifact /bin/bash -c "cp /build/artifacts/*.zip /output/artifacts/" + build: @set -euo pipefail cd ../../.. diff --git a/build/sbt.sh b/build/sbt.sh new file mode 100644 index 000000000..730b36b61 --- /dev/null +++ b/build/sbt.sh @@ -0,0 +1,7 @@ +#!/usr/bin/env bash +set -euo pipefail + +DIR=$(cd -P -- "$(dirname -- "$0")" && pwd -P) + +cd "$DIR/../test/api/integration" +make build DOCKER_PARAMS="--build-arg SKIP_API_BUILD=true" && make run-local WITH_ARGS="sbt" DOCKER_PARAMS="-e RUN_SERVICES=none" diff --git a/modules/api/src/main/scala/vinyldns/api/Boot.scala b/modules/api/src/main/scala/vinyldns/api/Boot.scala index 8214d263d..b9a989024 100644 --- a/modules/api/src/main/scala/vinyldns/api/Boot.scala +++ b/modules/api/src/main/scala/vinyldns/api/Boot.scala @@ -228,5 +228,7 @@ object Boot extends App { case Left(startupFailure) => logger.error(s"VINYLDNS SERVER UNABLE TO START $startupFailure") startupFailure.printStackTrace() + // It doesn't do us much good to keep the application running if it failed to start. + sys.exit(1) } } diff --git a/modules/api/src/main/scala/vinyldns/api/backend/dns/DnsConversions.scala b/modules/api/src/main/scala/vinyldns/api/backend/dns/DnsConversions.scala index c27646b5b..b9953d3a7 100644 --- a/modules/api/src/main/scala/vinyldns/api/backend/dns/DnsConversions.scala +++ b/modules/api/src/main/scala/vinyldns/api/backend/dns/DnsConversions.scala @@ -17,8 +17,8 @@ package vinyldns.api.backend.dns import java.net.InetAddress - import cats.syntax.either._ +import org.apache.commons.codec.binary.Hex import org.joda.time.DateTime import org.xbill.DNS import scodec.bits.ByteVector @@ -37,9 +37,9 @@ trait DnsConversions { import DomainHelpers._ /** - * record names are either relative to the zone name or - * if the record name is '@' the zone name is used - */ + * record names are either relative to the zone name or + * if the record name is '@' the zone name is used + */ def relativize(r: DNS.Name, zoneName: DNS.Name): String = if (r.equals(zoneName)) zoneName.toString @@ -170,15 +170,15 @@ trait DnsConversions { } /** - * Converts the list of raw DNS records to a list of record sets. - * - * Will join / combine DNS.Records that belong in the same record set. - */ + * Converts the list of raw DNS records to a list of record sets. + * + * Will join / combine DNS.Records that belong in the same record set. + */ def toFlattenedRecordSets( - records: List[DNS.Record], - zoneName: DNS.Name, - zoneId: String = "unknown" - ): List[RecordSet] = { + records: List[DNS.Record], + zoneName: DNS.Name, + zoneId: String = "unknown" + ): List[RecordSet] = { /* Combines record sets into a list of one or Nil in case there are no record sets in the list provided */ def combineRecordSets(lst: List[RecordSet]): RecordSet = @@ -195,7 +195,7 @@ trait DnsConversions { // Do a "relativize" using the zoneName, this removes the zone name from the record itself // For example "test-01.vinyldns." becomes "test-01"...this is necessary as we want to run comparisons upstream def fromDnsRecord[A <: DNS.Record](r: A, zoneName: DNS.Name, zoneId: String)( - f: A => List[RecordData] + f: A => List[RecordData] ): RecordSet = record.RecordSet( zoneId = zoneId, @@ -302,7 +302,7 @@ trait DnsConversions { def fromSSHFPRecord(r: DNS.SSHFPRecord, zoneName: DNS.Name, zoneId: String): RecordSet = fromDnsRecord(r, zoneName, zoneId) { data => - List(SSHFPData(data.getAlgorithm, data.getDigestType, new String(data.getFingerPrint))) + List(SSHFPData(data.getAlgorithm, data.getDigestType, Hex.encodeHexString(data.getFingerPrint).toUpperCase)) } def fromTXTRecord(r: DNS.TXTRecord, zoneName: DNS.Name, zoneId: String): RecordSet = @@ -390,7 +390,7 @@ trait DnsConversions { ) case SSHFPData(algorithm, typ, fingerprint) => - new DNS.SSHFPRecord(recordName, DNS.DClass.IN, ttl, algorithm, typ, fingerprint.getBytes) + new DNS.SSHFPRecord(recordName, DNS.DClass.IN, ttl, algorithm, typ, Hex.decodeHex(fingerprint.toCharArray())) case SPFData(text) => new DNS.SPFRecord(recordName, DNS.DClass.IN, ttl, text) @@ -432,10 +432,10 @@ trait DnsConversions { } def toUpdateRecordMessage( - r: DNS.RRset, - old: DNS.RRset, - zoneName: String - ): Either[Throwable, DNS.Update] = { + r: DNS.RRset, + old: DNS.RRset, + zoneName: String + ): Either[Throwable, DNS.Update] = { val update = new DNS.Update(zoneDnsName(zoneName)) if (!r.getName.equals(old.getName) || r.getTTL != old.getTTL) { // Name or TTL has changed diff --git a/modules/api/src/test/functional/tests/test_data.py b/modules/api/src/test/functional/tests/test_data.py index 0163f4027..f1f545cae 100644 --- a/modules/api/src/test/functional/tests/test_data.py +++ b/modules/api/src/test/functional/tests/test_data.py @@ -102,8 +102,8 @@ class TestData: "records": [ { "algorithm": 1, - "type": 2, - "fingerprint": "fp" + "type": 1, + "fingerprint": "123456789ABCDEF67890123456789ABCDEF67890" } ] } diff --git a/modules/api/src/test/scala/vinyldns/api/backend/CommandHandlerSpec.scala b/modules/api/src/test/scala/vinyldns/api/backend/CommandHandlerSpec.scala index 22f5d63c6..1298b3b27 100644 --- a/modules/api/src/test/scala/vinyldns/api/backend/CommandHandlerSpec.scala +++ b/modules/api/src/test/scala/vinyldns/api/backend/CommandHandlerSpec.scala @@ -309,7 +309,7 @@ class CommandHandlerSpec // verify our interactions verify(mq, atLeastOnce()).receive(count) - verify(mockRecordChangeProcessor) + verify(mockRecordChangeProcessor, atLeastOnce()) .apply(any[DnsBackend], mockito.Matchers.eq(pendingCreateAAAA)) verify(mq).remove(cmd) } diff --git a/modules/api/src/test/scala/vinyldns/api/backend/dns/DnsConversionsSpec.scala b/modules/api/src/test/scala/vinyldns/api/backend/dns/DnsConversionsSpec.scala index 5a60c1c6f..bb8c06ae7 100644 --- a/modules/api/src/test/scala/vinyldns/api/backend/dns/DnsConversionsSpec.scala +++ b/modules/api/src/test/scala/vinyldns/api/backend/dns/DnsConversionsSpec.scala @@ -172,7 +172,7 @@ class DnsConversionsSpec RecordSetStatus.Active, DateTime.now, None, - List(SSHFPData(1, 2, "fingerprint")) + List(SSHFPData(2, 1, "123456789ABCDEF67890123456789ABCDEF67890")) ) private val testTXT = RecordSet( testZone.id, diff --git a/modules/api/src/test/scala/vinyldns/api/domain/zone/ZoneConnectionValidatorSpec.scala b/modules/api/src/test/scala/vinyldns/api/domain/zone/ZoneConnectionValidatorSpec.scala index 84fe50dd7..10e3e53ac 100644 --- a/modules/api/src/test/scala/vinyldns/api/domain/zone/ZoneConnectionValidatorSpec.scala +++ b/modules/api/src/test/scala/vinyldns/api/domain/zone/ZoneConnectionValidatorSpec.scala @@ -73,7 +73,7 @@ class ZoneConnectionValidatorSpec List(new Regex("some.test.ns.")), 10000 ) { - override val opTimeout: FiniteDuration = 10.milliseconds + override val opTimeout: FiniteDuration = 60.seconds override def loadDns(zone: Zone): IO[ZoneView] = testLoadDns(zone) override def isValidBackendId(backendId: Option[String]): Either[Throwable, Unit] = Right(()) diff --git a/modules/portal/.gitignore b/modules/portal/.gitignore index 843786a95..a12c6d68c 100644 --- a/modules/portal/.gitignore +++ b/modules/portal/.gitignore @@ -10,6 +10,7 @@ release.version private .bloop .metals +run.sh public/js/* !public/js/custom.js diff --git a/modules/portal/README.md b/modules/portal/README.md index b1ccca194..02ddd7825 100644 --- a/modules/portal/README.md +++ b/modules/portal/README.md @@ -4,7 +4,7 @@ Supplies a UI for and offers authentication into VinylDNS. # Running Unit Tests -First, startup sbt: `sbt`. +First, startup sbt: `build/sbt.sh`. Next, you can run all tests by simply running `test`, or you can run an individual test by running `test-only *MySpec` diff --git a/modules/portal/karma.conf.js b/modules/portal/karma.conf.js index ac09db9d3..6f18b11c1 100644 --- a/modules/portal/karma.conf.js +++ b/modules/portal/karma.conf.js @@ -5,66 +5,76 @@ const puppeteer = require('puppeteer'); process.env.CHROME_BIN = puppeteer.executablePath(); module.exports = function(config) { - config.set({ - // base path, that will be used to resolve files and exclude - basePath: 'public/', + config.set({ + // base path, that will be used to resolve files and exclude + basePath: 'public/', - // testing framework to use (jasmine/mocha/qunit/...) - frameworks: ['jasmine'], + // testing framework to use (jasmine/mocha/qunit/...) + frameworks: ['jasmine'], - // list of files / patterns to load in the browser - files: [ - '*.js', - //fixtures - {pattern: 'mocks/*.json', watched: true, served: true, included: false} - ], + // list of files / patterns to load in the browser + files: [ + 'js/jquery.min.js', + 'js/bootstrap.min.js', + 'js/angular.min.js', + 'js/moment.min.js', + 'js/ui.js', + 'test_frameworks/*.js', + 'js/vinyldns.js', + 'lib/services/**/*.spec.js', + 'lib/controllers/**/*.spec.js', + 'lib/directives/**/*.spec.js', + 'lib/*.js', + //fixtures + {pattern: 'mocks/*.json', watched: true, served: true, included: false}, + ], - // list of files / patterns to exclude - exclude: [], + // list of files / patterns to exclude + exclude: [], - // web server port - port: 8080, + // web server port + port: 8080, - // level of logging - // possible values: LOG_DISABLE || LOG_ERROR || LOG_WARN || LOG_INFO || LOG_DEBUG - logLevel: config.LOG_INFO, + // level of logging + // possible values: LOG_DISABLE || LOG_ERROR || LOG_WARN || LOG_INFO || LOG_DEBUG + logLevel: config.LOG_INFO, - plugins: [ - 'karma-jasmine', - 'karma-chrome-launcher', - 'karma-mocha-reporter' - ], + plugins: [ + 'karma-jasmine', + 'karma-chrome-launcher', + 'karma-mocha-reporter', + ], - // reporter types: - // - dots - // - progress (default) - // - spec (karma-spec-reporter) - // - junit - // - growl - // - coverage - reporters: ['mocha'], + // reporter types: + // - dots + // - progress (default) + // - spec (karma-spec-reporter) + // - junit + // - growl + // - coverage + reporters: ['mocha'], - // enable / disable watching file and executing tests whenever any file changes - autoWatch: true, + // enable / disable watching file and executing tests whenever any file changes + autoWatch: true, - // Start these browsers, currently available: - // - Chrome - // - ChromeCanary - // - Firefox - // - Opera - // - Safari (only Mac) - // - PhantomJS - // - IE (only Windows) - browsers: ['ChromeHeadlessNoSandbox'], - customLaunchers: { - ChromeHeadlessNoSandbox: { - base: 'ChromeHeadless', - flags: ['--no-sandbox'] - } - }, + // Start these browsers, currently available: + // - Chrome + // - ChromeCanary + // - Firefox + // - Opera + // - Safari (only Mac) + // - PhantomJS + // - IE (only Windows) + browsers: ['ChromeHeadlessNoSandbox'], + customLaunchers: { + ChromeHeadlessNoSandbox: { + base: 'ChromeHeadless', + flags: ['--no-sandbox'], + }, + }, - // Continuous Integration mode - // if true, it capture browsers, run tests and exit - singleRun: true - }); + // Continuous Integration mode + // if true, it capture browsers, run tests and exit + singleRun: true, + }); }; diff --git a/modules/portal/public/lib/services/records/service.records.spec.js b/modules/portal/public/lib/services/records/service.records.spec.js index 4538c0178..4ba958fb8 100644 --- a/modules/portal/public/lib/services/records/service.records.spec.js +++ b/modules/portal/public/lib/services/records/service.records.spec.js @@ -101,16 +101,16 @@ describe('Service: recordsService', function () { "name": 'recordName', "type": 'SSHFP', "ttl": '300', - "sshfpItems": [{algorithm: '1', type: '1', fingerprint: 'foo'}, - {algorithm: '2', type: '1', fingerprint: 'bar'}] + "sshfpItems": [{algorithm: '1', type: '1', fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: '2', type: '1', fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}] }; expectedRecord = { "id": 'recordId', "name": 'recordName', "type": 'SSHFP', "ttl": 300, - "records": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}] + "records": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}] }; var actualRecord = this.recordsService.toVinylRecord(sentRecord); @@ -123,8 +123,8 @@ describe('Service: recordsService', function () { "name": 'recordName', "type": 'SSHFP', "ttl": 300, - "records": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}] + "records": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}] }; displayRecord = { @@ -133,8 +133,8 @@ describe('Service: recordsService', function () { "type": 'SSHFP', "ttl": 300, "records": undefined, - "sshfpItems": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}], + "sshfpItems": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}], "onlyFour": true, "isDotted": false, "canBeEdited": true @@ -150,8 +150,8 @@ describe('Service: recordsService', function () { "name": 'recordName.with.dot', "type": 'SSHFP', "ttl": 300, - "records": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}] + "records": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}] }; displayRecord = { @@ -160,8 +160,8 @@ describe('Service: recordsService', function () { "type": 'SSHFP', "ttl": 300, "records": undefined, - "sshfpItems": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}], + "sshfpItems": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}], "onlyFour": true, "isDotted": true, "canBeEdited": true @@ -177,8 +177,8 @@ describe('Service: recordsService', function () { "name": 'apex.with.dot', "type": 'SSHFP', "ttl": 300, - "records": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}] + "records": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}] }; displayRecord = { @@ -187,8 +187,8 @@ describe('Service: recordsService', function () { "type": 'SSHFP', "ttl": 300, "records": undefined, - "sshfpItems": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}], + "sshfpItems": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}], "onlyFour": true, "isDotted": false, "canBeEdited": true @@ -229,8 +229,8 @@ describe('Service: recordsService', function () { "name": 'apex.with.dot', "type": 'SSHFP', "ttl": 300, - "records": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}] + "records": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}] }; displayRecord = { @@ -239,8 +239,8 @@ describe('Service: recordsService', function () { "type": 'SSHFP', "ttl": 300, "records": undefined, - "sshfpItems": [{algorithm: 1, type: 1, fingerprint: 'foo'}, - {algorithm: 2, type: 1, fingerprint: 'bar'}], + "sshfpItems": [{algorithm: 1, type: 1, fingerprint: '123456789ABCDEF67890123456789ABCDEF67890'}, + {algorithm: 2, type: 1, fingerprint: 'F23456789ABCDEF67890123456789ABCDEF67890'}], "onlyFour": true, "isDotted": false, "canBeEdited": true diff --git a/modules/portal/run_all_tests.sh b/modules/portal/run_all_tests.sh deleted file mode 100755 index 808512f0a..000000000 --- a/modules/portal/run_all_tests.sh +++ /dev/null @@ -1,39 +0,0 @@ -#!/bin/bash - -function check_for() { - which $1 >/dev/null 2>&1 - EXIT_CODE=$? - if [ ${EXIT_CODE} != 0 ] - then - echo "$1 is not installed" - exit ${EXIT_CODE} - fi -} - -check_for python -check_for npm - -# if the program exits before this has been captured then there must have been an error -EXIT_CODE=1 - -cd $(dirname $0) - -# javascript code generate -bower install -grunt default - -TEST_SUITES=('sbt clean coverage test' - 'grunt unit' - ) - -for TEST in "${TEST_SUITES[@]}" -do - echo "##### Running test: [$TEST]" - $TEST - EXIT_CODE=$? - echo "##### Test [$TEST] ended with status [$EXIT_CODE]" - if [ ${EXIT_CODE} != 0 ] - then - exit ${EXIT_CODE} - fi -done diff --git a/modules/portal/test/controllers/VinylDNSSpec.scala b/modules/portal/test/controllers/VinylDNSSpec.scala index 43af81938..c53b82a27 100644 --- a/modules/portal/test/controllers/VinylDNSSpec.scala +++ b/modules/portal/test/controllers/VinylDNSSpec.scala @@ -573,17 +573,6 @@ class VinylDNSSpec extends Specification with Mockito with TestApplicationData w } } - ".oidcCallback" should { - "redirect to set session view" in new WithApplication(app) { - val response = vinyldnsPortal - .oidcCallback("id") - .apply(FakeRequest("GET", "id?query=q")) - - status(response) mustEqual 200 - contentAsString(response) must contain("/public/lib/oidc-finish.js") - } - } - ".newGroup" should { tag("slow") "return the group description on create - status ok (200)" in new WithApplication(app) { diff --git a/quickstart/README.md b/quickstart/README.md index c23750746..fb9636c6f 100644 --- a/quickstart/README.md +++ b/quickstart/README.md @@ -22,16 +22,17 @@ From a shell in the `quickstart/` directory, simply run: ```shell script ./quickstart-vinyldns.sh ``` + The `quickstart-vinyldns.sh` script takes a number of optional arguments: -| Flag | Description | -|:---|:---| -| `-a, --api-only` | do not start up the VinylDNS Portal" | -| `-b, --build` | force a rebuild of the Docker images with the local code" | -| `-c, --clean` | stops all VinylDNS containers" | -| `-d, --deps-only` | only start up the dependencies, not the API or Portal" | -| `-r, --reset` | reset any the running containers" | -| `-s, --service` | specify the service to run" | +| Flag | Description | +|:---|:-----------------------------------------------------------------------------| +| `-a, --api-only` | do not start up the VinylDNS Portal" | +| `-b, --build` | force a rebuild of the Docker images with the local code" | +| `-c, --clean` | stops all VinylDNS containers" | +| `-d, --deps-only` | only start up the dependencies, not the API or Portal" | +| `-r, --reset` | reset any the running containers" | +| `-s, --service` | specify the service to run" | | `-t, --timeout` | the time to wait (in seconds) for the Portal and API to start (default: 60)" | -| `-u, --update` | remove the local quickstart images to force a re-pull from Docker Hub" | -| `-v, --version-tag` | specify Docker image tag version (default: latest)" | +| `-u, --update` | remove the local quickstart images to force a re-pull from Docker Hub" | +| `-v, --version-tag` | specify Docker image tag version (default: latest)" | diff --git a/quickstart/docker-compose.yml b/quickstart/docker-compose.yml index 5460696dc..61f9e615c 100644 --- a/quickstart/docker-compose.yml +++ b/quickstart/docker-compose.yml @@ -1,12 +1,15 @@ version: "3.5" services: + + # LDAP container hosting example users ldap: container_name: "vinyldns-ldap" image: vinyldns/build:openldap ports: - "19004:19004" + # Integration image hosting r53, sns, sqs, bind, and mysql integration: container_name: "vinyldns-api-integration" hostname: &integration_hostname "vinyldns-integration" @@ -25,6 +28,7 @@ services: - "19001-19003:19001-19003/tcp" - "19001:19001/udp" + # The VinylDNS API api: container_name: "vinyldns-api" image: "vinyldns/api:${VINYLDNS_IMAGE_VERSION}" @@ -35,7 +39,7 @@ services: VINYLDNS_VERSION: "${VINYLDNS_IMAGE_VERSION}" DOCKER_FILE_PATH: "../build/docker/api" volumes: - - ../build/docker/api/application.conf:/opt/vinyldns/conf/vinyldns.conf + - ../build/docker/api/application.conf:/opt/vinyldns/conf/application.conf env_file: .env ports: @@ -43,6 +47,7 @@ services: depends_on: - integration + # The VinylDNS portal portal: container_name: "vinyldns-portal" image: "vinyldns/portal:${VINYLDNS_IMAGE_VERSION}" @@ -62,6 +67,7 @@ services: - api - ldap +# Custom network so that we don't interfere with the host system networks: default: name: "vinyldns_net" diff --git a/test/api/functional/Dockerfile b/test/api/functional/Dockerfile index 058638172..d1298d291 100644 --- a/test/api/functional/Dockerfile +++ b/test/api/functional/Dockerfile @@ -1,16 +1,16 @@ # Build VinylDNS API if the JAR doesn't already exist FROM vinyldns/build:base-build as base-build ARG DOCKERFILE_PATH="test/api/functional" -COPY "${DOCKERFILE_PATH}/vinyldns.*" /opt/vinyldns/ +COPY "${DOCKERFILE_PATH}/application.conf" /opt/vinyldns/conf/ COPY . /build/ WORKDIR /build -## Run the build if we don't already have a vinyldns.jar -RUN if [ -f assembly/vinyldns.jar ]; then cp assembly/vinyldns.jar /opt/vinyldns; fi && \ - if [ ! -f /opt/vinyldns/vinyldns.jar ]; then \ +## Run the build if we don't already have a vinyldns-api.jar +RUN if [ -f artifacts/vinyldns-api.jar ]; then cp artifacts/vinyldns-api.jar /opt/vinyldns; fi && \ + if [ ! -f /opt/vinyldns/vinyldns-api.jar ]; then \ env SBT_OPTS="-XX:+UseConcMarkSweepGC -Xmx4G -Xms1G" \ sbt -Dbuild.scalafmtOnCompile=false -Dbuild.lintOnCompile=fase ";project api;coverageOff;assembly" \ - && cp modules/api/target/scala-2.12/vinyldns.jar /opt/vinyldns/; \ + && cp artifacts/vinyldns-api.jar /opt/vinyldns/; \ fi # Build the testing image, copying data from `vinyldns-api` diff --git a/test/api/functional/Makefile b/test/api/functional/Makefile index 7704fcee6..b380ac9d2 100644 --- a/test/api/functional/Makefile +++ b/test/api/functional/Makefile @@ -2,7 +2,6 @@ SHELL=bash IMAGE_NAME=vinyldns-api-test ROOT_DIR:=$(shell dirname $(realpath $(lastword $(MAKEFILE_LIST)))) RELATIVE_ROOT_DIR:=$(shell realpath --relative-to=../../.. $(ROOT_DIR)) -VINYLDNS_JAR_PATH?=modules/api/target/scala-2.12/vinyldns.jar # Check that the required version of make is being used REQ_MAKE_VER:=3.82 @@ -28,7 +27,7 @@ endif .ONESHELL: -.PHONY: all build run run-local +.PHONY: all build run run-local run-deps-bg clean-containers all: build run @@ -42,7 +41,19 @@ run: USE_TTY="" && test -t 1 && USE_TTY="-t" docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) -p 9000:9000 -p 19001-19003:19001-19003 -p 19001:19001/udp $(IMAGE_NAME) $(ARG_SEPARATOR) $(WITH_ARGS) +# Runs the dependencies for the functional test in the background +# This is useful when running the tests on your host machine against the API in a container +run-deps-bg: + @set -euo pipefail + docker stop $(IMAGE_NAME) &> /dev/null || true + USE_TTY="" && test -t 1 && USE_TTY="-t" + docker run -d $${USE_TTY} --name $(IMAGE_NAME) --rm $(DOCKER_PARAMS) --entrypoint "/initialize.sh" -p 9000:9000 -p 19001-19003:19001-19003 -p 19001:19001/udp $(IMAGE_NAME) all tail-logs + run-local: @set -euo pipefail USE_TTY="" && test -t 1 && USE_TTY="-t" docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) -p 9000:9000 -p 19001-19003:19001-19003 -p 19001:19001/udp -v "$(ROOT_DIR)/../../../modules/api/src/test/functional:/functional_test" $(IMAGE_NAME) -- $(WITH_ARGS) + +clean-containers: + @set -euo pipefail + "$(ROOT_DIR)/../../../utils/clean-vinyldns-containers.sh" diff --git a/test/api/functional/vinyldns.conf b/test/api/functional/application.conf similarity index 100% rename from test/api/functional/vinyldns.conf rename to test/api/functional/application.conf diff --git a/test/api/integration/Dockerfile b/test/api/integration/Dockerfile index ae0241bfd..3f7169d49 100644 --- a/test/api/integration/Dockerfile +++ b/test/api/integration/Dockerfile @@ -3,16 +3,17 @@ ARG VINYLDNS_BASE_VERSION=latest # Build VinylDNS API if the JAR doesn't already exist FROM vinyldns/build:base-build as base-build ARG DOCKERFILE_PATH="test/api/integration" -COPY "${DOCKERFILE_PATH}/vinyldns.*" /opt/vinyldns/ +COPY "${DOCKERFILE_PATH}/application.conf" /opt/vinyldns/conf/ COPY . /build/ WORKDIR /build -## Run the build if we don't already have a vinyldns.jar -RUN if [ -f assembly/vinyldns.jar ]; then cp assembly/vinyldns.jar /opt/vinyldns; fi && \ - if [ ! -f /opt/vinyldns/vinyldns.jar ]; then \ +## Run the build if we don't already have a vinyldns-api.jar +ARG SKIP_API_BUILD="false" +RUN if [ -f artifacts/vinyldns-api.jar ]; then cp artifacts/vinyldns-api.jar /opt/vinyldns; fi && \ + if [ ! -f /opt/vinyldns/vinyldns-api.jar ] && [ "$SKIP_API_BUILD" == "false" ]; then \ env SBT_OPTS="-XX:+UseConcMarkSweepGC -Xmx4G -Xms1G" \ sbt -Dbuild.scalafmtOnCompile=false -Dbuild.lintOnCompile=fase ";project api;coverageOff;assembly" \ - && cp assembly/vinyldns.jar /opt/vinyldns/; \ + && cp artifacts/vinyldns-api.jar /opt/vinyldns/; \ fi # Build the testing image, copying data from `base-build` diff --git a/test/api/integration/Makefile b/test/api/integration/Makefile index 0ee3b5d3b..1bbb79ff8 100644 --- a/test/api/integration/Makefile +++ b/test/api/integration/Makefile @@ -27,14 +27,14 @@ endif .ONESHELL: -.PHONY: all build run run-local +.PHONY: all build run run-local run-bg stop-bg clean-containers all: build run build: @set -euo pipefail cd ../../.. - docker build -t $(IMAGE_NAME) --build-arg DOCKERFILE_PATH="$(RELATIVE_ROOT_DIR)" -f "$(ROOT_DIR)/Dockerfile" . + docker build -t $(IMAGE_NAME) $(DOCKER_PARAMS) --build-arg DOCKERFILE_PATH="$(RELATIVE_ROOT_DIR)" -f "$(ROOT_DIR)/Dockerfile" . run: @set -euo pipefail @@ -54,4 +54,8 @@ stop-bg: run-local: @set -euo pipefail USE_TTY="" && test -t 1 && USE_TTY="-t" - docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) -p 9000:9000 -p 19001-19003:19001-19003 -p 19001:19001/udp -v "$(ROOT_DIR)/../../..:/build" $(IMAGE_NAME) -- $(WITH_ARGS) + docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) -v "$(ROOT_DIR)/../../..:/build" $(IMAGE_NAME) -- $(WITH_ARGS) + +clean-containers: + @set -euo pipefail + "$(ROOT_DIR)/../../../utils/clean-vinyldns-containers.sh" diff --git a/test/api/integration/vinyldns.conf b/test/api/integration/application.conf similarity index 100% rename from test/api/integration/vinyldns.conf rename to test/api/integration/application.conf diff --git a/test/portal/functional/Makefile b/test/portal/functional/Makefile index 140456856..59f3f6cd1 100644 --- a/test/portal/functional/Makefile +++ b/test/portal/functional/Makefile @@ -18,16 +18,13 @@ ifeq ($(EXTRACT_ARGS),true) # use the rest as arguments for "run" WITH_ARGS ?= $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS)) endif -ifneq ($(WITH_ARGS),) - ARG_SEPARATOR=-- -endif %: @: .ONESHELL: -.PHONY: all build run run-local +.PHONY: all build run run-local clean-containers all: build run @@ -39,9 +36,13 @@ build: run: @set -euo pipefail USE_TTY="" && test -t 1 && USE_TTY="-t" - docker run -i $${USE_TTY} --rm $(IMAGE_NAME) -- $(WITH_ARGS) + docker run -i $${USE_TTY} --rm $(IMAGE_NAME) $(WITH_ARGS) run-local: @set -euo pipefail USE_TTY="" && test -t 1 && USE_TTY="-t" - docker run -i $${USE_TTY} --rm -v "$$(pwd)/../../../modules/portal:/functional_test" $(IMAGE_NAME) $(ARG_SEPARATOR) $(WITH_ARGS) + docker run -i $${USE_TTY} --rm -v "$$(pwd)/../../../modules/portal:/functional_test" -v "$$(pwd)/run.sh:/functional_test/run.sh" $(IMAGE_NAME) $(WITH_ARGS) + +clean-containers: + @set -euo pipefail + "$(ROOT_DIR)/../../../utils/clean-vinyldns-containers.sh" diff --git a/test/portal/functional/run.sh b/test/portal/functional/run.sh index 753d0e239..c0fad0e32 100755 --- a/test/portal/functional/run.sh +++ b/test/portal/functional/run.sh @@ -3,11 +3,13 @@ set -eo pipefail ROOT_DIR=$(cd -P -- "$(dirname -- "$0")" && pwd -P) - cd "${ROOT_DIR}" if [ "$1" == "--interactive" ]; then shift bash else - grunt unit "$@" + # Attempt to just run grunt - this should work most of the time + # We may need to update dependencies if our local functional tests dependencies + # differ from those of the 'base-test-portal' docker image + grunt unit "$@" || { echo "Attempting to recover.." && npm install -f --no-audit --no-fund && grunt unit "$@"; } fi diff --git a/utils/admin/Dockerfile b/utils/admin/Dockerfile index 1701fac80..f292086af 100644 --- a/utils/admin/Dockerfile +++ b/utils/admin/Dockerfile @@ -9,7 +9,7 @@ RUN mkdir -p /vinyldns/python && \ protoc --proto_path=/vinyldns --python_out=/vinyldns/python /vinyldns/VinylDNSProto.proto -FROM python:3.7-alpine +FROM vinyldns/build:base-build ARG DOCKERFILE_PATH WORKDIR /app RUN pip install mysql-connector-python==8.0.27 diff --git a/utils/admin/update-support-user.py b/utils/admin/update-support-user.py index 70d58a342..247ede022 100644 --- a/utils/admin/update-support-user.py +++ b/utils/admin/update-support-user.py @@ -1,8 +1,9 @@ #!/usr/bin/env python -import mysql.connector -import VinylDNSProto_pb2 -import sys import os +import sys + +import VinylDNSProto_pb2 +import mysql.connector # arguments if len(sys.argv) != 3: @@ -27,7 +28,7 @@ update = "UPDATE user SET data = %(pb)s WHERE user_name = %(user_name)s" cursor = cnx.cursor(dictionary=True) try: - cursor.execute(query, { 'user_name': user_name }) + cursor.execute(query, {'user_name': user_name}) user = VinylDNSProto_pb2.User() for row in cursor: user_raw = row['data'] @@ -46,7 +47,7 @@ try: if make_support is not None: print("Updating {}, Support = {}".format(user_name, make_support)) user.isSupport = make_support - cursor.execute(update, { 'pb': user.SerializeToString(), 'user_name': user_name }) + cursor.execute(update, {'pb': user.SerializeToString(), 'user_name': user_name}) cnx.commit() else: print("Skipping making support as no make support value provided") diff --git a/utils/update-support-user.sh b/utils/update-support-user.sh index c08269a9e..976b60f9c 100755 --- a/utils/update-support-user.sh +++ b/utils/update-support-user.sh @@ -1,32 +1,26 @@ #!/usr/bin/env bash usage () { - echo -e "Description: Updates a user in VinylDNS to a support user, or removes the user as a support user.\n" echo -e "Usage: update-support-user.sh [OPTIONS] \n" + echo -e "Description: Updates a user in VinylDNS to a support user, or removes the user as a support user.\n" echo -e "Required Parameters:" echo -e "username\tThe VinylDNS user for which to change the support flag" echo -e "enableSupport\t'true' to set the user as a support user; 'false' to remove support privileges\n" echo -e "OPTIONS:" - echo -e "Must define as an environment variables the following (or pass them in on the command line)\n" - echo -e "DB_USER (user name for accessing the VinylDNS database)" - echo -e "DB_PASS (user password for accessing the VinylDNS database)" - echo -e "DB_HOST (host name for the mysql server of the VinylDNS database)" - echo -e "DB_NAME (name of the VinylDNS database, defaults to vinyldns)" - echo -e "DB_PORT (port of the VinylDNS database, defaults to 19002)\n" - echo -e " -u|--user \tDatabase user name for accessing the VinylDNS database" - echo -e " -p|--password\tDatabase user password for accessing the VinylDNS database" - echo -e " -h|--host\tDatabase host name for the mysql server" - echo -e " -n|--name\tName of the VinylDNS database, defaults to vinyldns" - echo -e " -c|--port\tPort of the VinylDNS database, defaults to 19002" + echo -e " -u|--user \tDatabase user name for accessing the VinylDNS database (DB_USER - default=root)" + echo -e " -p|--password\tDatabase user password for accessing the VinylDNS database (DB_PASS - default=pass)" + echo -e " -h|--host\tDatabase host name for the mysql server (DB_HOST - default=vinyldns-integration)" + echo -e " -n|--name\tName of the VinylDNS database, (DB_NAME - default=vinyldns)" + echo -e " -c|--port\tPort of the VinylDNS database, (DB_PORT - default=19002)" } DIR=$( cd "$(dirname "$0")" || exit ; pwd -P ) VINYL_ROOT=$DIR/.. WORK_DIR=${VINYL_ROOT}/docker -DB_USER=$DB_USER -DB_PASS=$DB_PASS -DB_HOST=$DB_HOST +DB_USER=${DB_USER:-root} +DB_PASS=${DB_PASS:-pass} +DB_HOST=${DB_HOST:-vinyldns-integration} DB_NAME=${DB_NAME:-vinyldns} DB_PORT=${DB_PORT:-19002} @@ -46,50 +40,36 @@ VINYL_USER="$1" MAKE_SUPPORT="$2" ERROR= -if [[ -z "$DB_USER" ]] -then - echo "No DB_USER environment variable found" +if [ -z "$DB_USER" ]; then ERROR="1" fi -if [[ -z "$DB_PASS" ]] -then - echo "No DB_PASS environment variable found" +if [ -z "$DB_PASS" ]; then ERROR="1" fi -if [[ -z "$DB_HOST" ]] -then - echo "No DB_HOST environment variable found" +if [ -z "$DB_HOST" ]; then ERROR="1" fi -if [[ -z "$DB_NAME" ]] -then - echo "No DB_NAME environment variable found" +if [ -z "$DB_NAME" ]; then ERROR="1" fi - -if [[ -z "$VINYL_USER" ]] -then - echo "Parameter 'username' not specified" +if [ -z "$VINYL_USER" ]; then ERROR="1" fi -if [[ -z "$MAKE_SUPPORT" ]] -then - echo "Parameter 'enableSupport' not specified" +if [ -z "$MAKE_SUPPORT" ]; then ERROR="1" fi -if [[ -n "$ERROR" ]] -then +if [ -n "$ERROR" ]; then usage exit 1 fi -# Copy the proto definition to the Docker context and build +# Build and run the Docker container cd admin make build make run DOCKER_PARAMS="-e \"DB_USER=$DB_USER\" -e \"DB_PASS=$DB_PASS\" -e \"DB_HOST=$DB_HOST\" -e \"DB_NAME=$DB_NAME\" -e \"DB_PORT=$DB_PORT\"" WITH_ARGS="\"$VINYL_USER\" \"$MAKE_SUPPORT\""