Major overhaul of func tests to allow them to run in parallel. Major changes include:
1. Consolidate all separate test fixtures into a single test fixture in the `shared_zone_test_context`
1. Add `xdist` to allow running tests in parallel
1. Add hooks in main `conftest.py` to setup the test fixture before workers run, and tear it down when workers are finished
1. After fixture is setup, save state in a local `tmp.out` so the workers will use that state instead of trying to recreate the fixture.
1. Add a `utils.generate_record_name` which generates a unique record name in order to avoid conflicts when running tests in parallel
1. Add a `pytest.mark.serial` for func tests that just cannot be run in serial
1. Tests are now run in two phases, first we run in parallel, and if that is successful, we run the serial tests
1. Add a `--teardown` flag, this allows us to reuse the test fixture between the two phases parallel and serial
* Fix docker releases
There was an issue starting the docker containers due to how native packager
works where we were seeing issues with the container being able to start.
The issue was that we were assuming a "daemon" user to run the containers under.
At some point this changed to "1001:0". As a result, there were not sufficient
privileges to start the containers because the "daemon" user was invalid or did
not have access to the scripts created by sbt native packager.
* `build.sbt` - update the user to "1001:0" for our custom install. Cleaned up the hardcoded references in the script extras to `/opt/docker` to use the variable `app_home` instead.
* `plugins.sbt` - updated to the latest sbt native packager
* Add MySqlRecordSetRepository
* Updated docker for mysql to use general_log for fun sql debug times
* Made sure to use rewriteBatchStatements to acheive new hights for bulk inserts
* `MySqlDataStoreProvider` support for the record set repo
* Externalize mysql.
* Updates based on feedback (pauljamescleary).
* Update publish settings for mysql.
* WIP
* Updates based on feedback (rebstar6).
* Update reference to MySQL.
* Use config file for API integration tests.
* Fixed scalikejdbc version.
* Add back application.conf
* Be more specific with MySQL settings.
* Update test config for MySQL module.
* Updates based on feedback (rebstar6).
Using the `all` command in sbt allows us to run certain tasks
in parallel. Not everything can be done in parallel, so we have to use
judgement here. Some things like dockerCompose cannot be done in parallel.
With the adjustments, local `;validate;verify` went from 12 minutes to 9
minutes, a savings of 3 full minutes!
**build.sbt**
Most changes are here.
Allow parallelExecution by default. The reason is that test suites will be run
in parallel. The play framework by default turns this off. Some of the changes
made in here around the `InMemoryBatchChangeRepository` were necessary when I
flipped this on.
We cannot run in parallel by default for IntegrationTest. The reason is that
several of the api integration tests use the same zone repo, so they wind up
stomping on each other.
Added a `killDocker` task that is much faster to run than `dockerComposeStop`
Enabled parallelExecution in IntegrationTest in dynamodb. The integration tests
do not conflict with each other here.
Changed the command aliases to use parallel `all`
**InMemoryBatchCHangeRepository**
This used to be a singleton, which prevents running unit tests in parallel.
Made this a class, and updated unit tests to use the class instead.
**logback-test.xml**
We were still logging in odd ways in places. Removed this to turn logging off
by default for tests.
**.jvmopts**
Kept running out of metaspace. Increased the memory needed to help slow that
down.
* Fixing portal build
The dynamodb sub module is not automatically packaged with the
portal universal distribution.
Need to modify sbt to make sure it is.
* Setting organization on dynamodb
Since this will be published to sonatype, has to be io.vinyldns
* Quiet build output
* set the traceLevel in the build to -1, which means no exceptions
will be output when running
* set the akka.logLevel in a few files to `OFF` so we don't
log anything during tests
* Quiet portal exceptions
Replace the repos in the portal with dynamodb and core
* Remove all data stores from the portal
* Use the user and user change repository from core and dynamodb
* Remove the UserAccount type, use core User instead
* Remove the UserChangeLog types, use core UserChange instead
* Clean up duplication in VinylDNS
* Moved `Module` to `modules.VinylDNSModule`. The reason is that
you cannot disable the "default" module for unit tests.
* Use mock configuration for VinylDNSSpec and FrontendControllerSpec.
The mock app configuration is what allows us to run without dynamodb
* Added a TestApplicationData trait to cut down on duplication
* IO startup for dynamodb stores (rather than unsafe throws)
* Update unit and integration tests in the dynamodb module
* update api module where dependent on dnamodb
* bin/release.sh script to check for required env variables, run tests, then run `sbt release`
* MAINTAINERS.md that describes steps needed to release
* implemented sbt release to run our release to docker and sonatype
The sbt protobuf plugin we were using forced developers to install
protoc version 2.6.1 on their local. That makes it difficult
to onboard new developers.
sbt-protoc is an alternative plugin. It has a lot of features we
are presently not using. The biggest feature it brings is to
not require developers to install protoc.
* build.sbt - use the new protoc plugin
* VinylDNSProto.proto - add syntax = proto2 to ensure compile
to compatible protobuf version
* plugins.sbt - remove the old protobuf plugin
* protoc.sbt - add the new protoc plugin
* Removing ProtocPlugin as it is an auto-plugin
* Remove uneeded protoc lib dependency
* Update docs to remove protoc requirement
* Remove protobuf install from travis
