mir/vinyldns
2
0
Fork 0
mirror of https://github.com/VinylDNS/vinyldns synced 2025-08-22 02:02:14 +00:00
Code Issues Releases Wiki Activity

Consistency Updates

Browse Source 
- Clean up documentation
- Update architecure diagram
- Fix discrepencies between local and docker test execution
- Fix inconsistencies in various configuration files used for tests and
execution
This commit is contained in:
Ryan Emerle 2021-12-08 14:36:00 -05:00
parent 9913b67d75
commit 85577a27f0
No known key found for this signature in database
GPG Key ID: C0D34C592AED41CE
36 changed files with 1597 additions and 722 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
AUTHORS.md
View File

@ -3,14 +3,9 @@
This project would not be possible without the generous contributions of many people. Thank you! If you have contributed This project would not be possible without the generous contributions of many people. Thank you! If you have contributed
in any way, but do not see your name here, please open a PR to add yourself (in alphabetical order by last name)! in any way, but do not see your name here, please open a PR to add yourself (in alphabetical order by last name)!
## DNS SMEs
- Joe Crowe
- David Back
- Hong Ye
## Contributors ## Contributors
- David Back
- Mike Ball - Mike Ball
- Tommy Barker - Tommy Barker
- Robert Barrimond - Robert Barrimond
@ -23,6 +18,7 @@ in any way, but do not see your name here, please open a PR to add yourself (in
- Peter Cline - Peter Cline
- Kemar Cockburn - Kemar Cockburn
- Luke Cori - Luke Cori
- Joe Crowe
- Jearvon Dharrie - Jearvon Dharrie
- Andrew Dunn - Andrew Dunn
- Ryan Emerle - Ryan Emerle
@ -54,3 +50,4 @@ in any way, but do not see your name here, please open a PR to add yourself (in
- Andrew Wang - Andrew Wang
- Peter Willis - Peter Willis
- Britney Wright - Britney Wright
- Hong Ye

168
DEVELOPER_GUIDE.md
View File

@ -2,10 +2,28 @@
## Table of Contents ## Table of Contents
- [Developer Requirements](#developer-requirements) - [Developer Requirements (Local)](#developer-requirements-local)
- [Developer Requirements (Docker)](#developer-requirements-docker)
- [Project Layout](#project-layout) - [Project Layout](#project-layout)
* [Core](#core)
* [API](#api)
* [Portal](#portal)
* [Documentation](#documentation)
- [Running VinylDNS Locally](#running-vinyldns-locally) - [Running VinylDNS Locally](#running-vinyldns-locally)
* [Starting the API Server](#starting-the-api-server)
* [Starting the Portal](#starting-the-portal)
- [Testing](#testing) - [Testing](#testing)
* [Unit Tests](#unit-tests)
* [Integration Tests](#integration-tests)
+ [Running both](#running-both)
* [Functional Tests](#functional-tests)
+ [Running Functional Tests](#running-functional-tests)
- [API Functional Tests](#api-functional-tests)
+ [Setup](#setup)
- [Functional Test Context](#functional-test-context)
- [Partitioning](#partitioning)
- [Really Important Test Context Rules!](#really-important-test-context-rules)
- [Managing Test Zone Files](#managing-test-zone-files)
## Developer Requirements (Local) ## Developer Requirements (Local)
@ -41,14 +59,14 @@ its components.
The main codebase is a multi-module Scala project with multiple sub-modules. To start working with the project, from the The main codebase is a multi-module Scala project with multiple sub-modules. To start working with the project, from the
root directory run `sbt`. Most of the code can be found in the `modules` directory. The following modules are present: root directory run `sbt`. Most of the code can be found in the `modules` directory. The following modules are present:
* `root` - this is the parent project, if you run tasks here, it will run against all sub-modules - `root` - this is the parent project, if you run tasks here, it will run against all sub-modules
* [`core`](#core): core modules that are used by both the API and portal, such as cryptography implementations. - [`core`](#core): core modules that are used by both the API and portal, such as cryptography implementations.
* [`api`](#api): the API is the main engine for all of VinylDNS. This is the most active area of the codebase, as - [`api`](#api): the API is the main engine for all of VinylDNS. This is the most active area of the codebase, as
everything else typically just funnels through the API. everything else typically just funnels through the API.
* [`portal`](#portal): The portal is a user interface wrapper around the API. Most of the business rules, logic, and - [`portal`](#portal): The portal is a user interface wrapper around the API. Most of the business rules, logic, and
processing can be found in the API. The processing can be found in the API. The
_only_ features in the portal not found in the API are creation of users and user authentication. _only_ features in the portal not found in the API are creation of users and user authentication.
* [`docs`](#documentation): documentation for VinylDNS. - [`docs`](#documentation): documentation for VinylDNS.
### Core ### Core
@ -56,76 +74,76 @@ Code that is used across multiple modules in the VinylDNS ecosystem live in `cor
#### Code Layout #### Code Layout
* `src/main` - the main source code - `src/main` - the main source code
* `src/test` - unit tests - `src/test` - unit tests
### API ### API
The API is the RESTful API for interacting with VinylDNS. The following technologies are used: The API is the RESTful API for interacting with VinylDNS. The following technologies are used:
* [Akka HTTP](https://doc.akka.io/docs/akka-http/current/) - Used primarily for REST and HTTP calls. - [Akka HTTP](https://doc.akka.io/docs/akka-http/current/) - Used primarily for REST and HTTP calls.
* [FS2](https://functional-streams-for-scala.github.io/fs2/) - Used for backend change processing off of message queues. - [FS2](https://functional-streams-for-scala.github.io/fs2/) - Used for backend change processing off of message queues.
FS2 has back-pressure built in, and gives us tools like throttling and concurrency. FS2 has back-pressure built in, and gives us tools like throttling and concurrency.
* [Cats Effect](https://typelevel.org/cats-effect/) - A replacement of `Future` with the `IO` monad - [Cats Effect](https://typelevel.org/cats-effect/) - A replacement of `Future` with the `IO` monad
* [Cats](https://typelevel.org/cats) - Used for functional programming. - [Cats](https://typelevel.org/cats) - Used for functional programming.
* [PureConfig](https://pureconfig.github.io/) - For loading configuration values. - [PureConfig](https://pureconfig.github.io/) - For loading configuration values.
The API has the following dependencies: The API has the following dependencies:
* MySQL - the SQL database that houses the data - MySQL - the SQL database that houses the data
* SQS - for managing concurrent updates and enabling high-availability - SQS - for managing concurrent updates and enabling high-availability
* Bind9 - for testing integration with a real DNS system - Bind9 - for testing integration with a real DNS system
#### Code Layout #### Code Layout
The API code can be found in `modules/api`. The API code can be found in `modules/api`.
* `src/it` - integration tests - `src/it` - integration tests
* `src/main` - the main source code - `src/main` - the main source code
* `src/test` - unit tests - `src/test` - unit tests
* `src/universal` - items that are packaged in the Docker image for the VinylDNS API - `src/universal` - items that are packaged in the Docker image for the VinylDNS API
The package structure for the source code follows: The package structure for the source code follows:
* `vinyldns.api.domain` - contains the core front-end logic. This includes things like the application services, - `vinyldns.api.domain` - contains the core front-end logic. This includes things like the application services,
repository interfaces, domain model, validations, and business rules. repository interfaces, domain model, validations, and business rules.
* `vinyldns.api.engine` - the back-end processing engine. This is where we process commands including record changes, - `vinyldns.api.engine` - the back-end processing engine. This is where we process commands including record changes,
zone changes, and zone syncs. zone changes, and zone syncs.
* `vinyldns.api.protobuf` - marshalling and unmarshalling to and from protobuf to types in our system - `vinyldns.api.protobuf` - marshalling and unmarshalling to and from protobuf to types in our system
* `vinyldns.api.repository` - repository implementations live here - `vinyldns.api.repository` - repository implementations live here
* `vinyldns.api.route` - HTTP endpoints - `vinyldns.api.route` - HTTP endpoints
### Portal ### Portal
The project is built using: The project is built using:
* [Play Framework](https://www.playframework.com/documentation/2.6.x/Home) - [Play Framework](https://www.playframework.com/documentation/2.6.x/Home)
* [AngularJS](https://angularjs.org/) - [AngularJS](https://angularjs.org/)
The portal is _mostly_ a shim around the API. Most actions in the user interface are translated into API calls. The portal is _mostly_ a shim around the API. Most actions in the user interface are translated into API calls.
The features that the Portal provides that are not in the API include: The features that the Portal provides that are not in the API include:
* Authentication against LDAP - Authentication against LDAP
* Creation of users - when a user logs in for the first time, VinylDNS automatically creates a user and new credentials - Creation of users - when a user logs in for the first time, VinylDNS automatically creates a user and new credentials
for them in the database with their LDAP information. for them in the database with their LDAP information.
#### Code Layout #### Code Layout
The portal code can be found in `modules/portal`. The portal code can be found in `modules/portal`.
* `app` - source code for portal back-end - `app` - source code for portal back-end
* `models` - data structures that are used by the portal - `models` - data structures that are used by the portal
* `views` - HTML templates for each web page - `views` - HTML templates for each web page
* `controllers` - logic for updating data - `controllers` - logic for updating data
* `conf` - configurations and endpoint routes - `conf` - configurations and endpoint routes
* `public` - source code for portal front-end - `public` - source code for portal front-end
* `css` - stylesheets - `css` - stylesheets
* `images` - images, including icons, used in the portal - `images` - images, including icons, used in the portal
* `js` - scripts - `js` - scripts
* `mocks` - mock JSON used in Grunt tests - `mocks` - mock JSON used in Grunt tests
* `templates` - modal templates - `templates` - modal templates
* `test` - unit tests for portal back-end - `test` - unit tests for portal back-end
### Documentation ### Documentation
@ -134,8 +152,8 @@ settings for the microsite are also configured in `build.sbt` of the project roo
#### Code Layout #### Code Layout
* `src/main/resources` - Microsite resources and configurations - `src/main/resources` - Microsite resources and configurations
* `src/main/mdoc` - Content for microsite web pages - `src/main/mdoc` - Content for microsite web pages
## Running VinylDNS Locally ## Running VinylDNS Locally
@ -146,20 +164,19 @@ README. However, VinylDNS can also be run in the foreground.
Before starting the API service, you can start the dependencies for local development: Before starting the API service, you can start the dependencies for local development:
``` ```shell
cd test/api/integration quickstart/quickstart-vinyldns.sh --deps-only
make build && make run-bg
``` ```
This will start a container running in the background with necessary prerequisites. This will start a container running in the background with necessary prerequisites.
Once the prerequisites are running, you can start up sbt by running `sbt` from the root directory. Once the prerequisites are running, you can start up sbt by running `sbt` from the root directory.
* `project api` to change the sbt project to the API - `project api` to change the sbt project to the API
* `reStart` to start up the API server - `reStart` to start up the API server
* Wait until you see the message `VINYLDNS SERVER STARTED SUCCESSFULLY` before working with the server - Wait until you see the message `VINYLDNS SERVER STARTED SUCCESSFULLY` before working with the server
* To stop the VinylDNS server, run `reStop` from the api project - To stop the VinylDNS server, run `reStop` from the api project
* To stop the dependent Docker containers: `utils/clean-vinyldns-containers.sh` - To stop the dependent Docker containers: `utils/clean-vinyldns-containers.sh`
See the [API Configuration Guide](https://www.vinyldns.io/operator/config-api) for information regarding API See the [API Configuration Guide](https://www.vinyldns.io/operator/config-api) for information regarding API
configuration. configuration.
@ -167,9 +184,9 @@ configuration.
### Starting the Portal ### Starting the Portal
To run the portal locally, you _first_ have to start up the VinylDNS API Server. This can be done by following the To run the portal locally, you _first_ have to start up the VinylDNS API Server. This can be done by following the
instructions for [Staring the API Server](#Starting the API Server) or by using the QuickStart: instructions for [Staring the API Server](#starting-the-api-server) or by using the QuickStart:
``` ```shell
quickstart/quickstart-vinyldns.sh --api-only quickstart/quickstart-vinyldns.sh --api-only
``` ```
@ -179,29 +196,6 @@ execute `;preparePortal; run`.
See the [Portal Configuration Guide](https://www.vinyldns.io/operator/config-portal) for information regarding portal See the [Portal Configuration Guide](https://www.vinyldns.io/operator/config-portal) for information regarding portal
configuration. configuration.
### Loading test data
Normally the portal can be used for all VinylDNS requests. Test users are locked down to only have access to test zones,
which the portal connection modal has not been updated to incorporate. To connect to a zone with testuser, you will need
to use an alternative client and set `isTest=true` on the zone being connected to.
Use the vinyldns-js client (Note, you need Node installed):
```
git clone https://github.com/vinyldns/vinyldns-js.git
cd vinyldns-js
npm install
export VINYLDNS_API_SERVER=http://localhost:9000
export VINYLDNS_ACCESS_KEY_ID=testUserAccessKey
export VINYLDNS_SECRET_ACCESS_KEY=testUserSecretKey
npm run repl
> var groupId;
> vinyl.createGroup({"name": "test-group", "email":"test@test.com", members: [{id: "testuser"}], admins: [{id: "testuser"}]}).then(res => {groupId = res.id}).catch(err => {console.log(err)});
> vinyl.createZone ({name: "ok.", isTest: true, adminGroupId: groupId, email: "test@test.com"}).then(res => { console.log(res) }).catch(err => { console.log(err) })
You should now be able to see the zone in the portal at localhost:9001 when logged in as username=testuser password=testpassword
```
## Testing ## Testing
### Unit Tests ### Unit Tests
@ -247,7 +241,7 @@ When adding new features, you will often need to write new functional tests that
To run functional tests you can simply execute the following commands: To run functional tests you can simply execute the following commands:
``` ```shell
build/func-test-api.sh build/func-test-api.sh
build/func-test-portal.sh build/func-test-portal.sh
``` ```
@ -259,9 +253,9 @@ These command will run the API functional tests and portal functional tests resp
To run functional tests you can simply execute `build/func-test-api.sh`, but if you'd like finer-grained control, you To run functional tests you can simply execute `build/func-test-api.sh`, but if you'd like finer-grained control, you
can work with the `Makefile` in `test/api/functional`: can work with the `Makefile` in `test/api/functional`:
``` ```shell
cd test/api/functional # Build and then run the function test container
make build && make run make -C test/api/functional build run
``` ```
During iterative test development, you can use `make run-local` which will bind-mount the current functional tests in During iterative test development, you can use `make run-local` which will bind-mount the current functional tests in
@ -289,20 +283,20 @@ There aren't a lot, so it should be quick.
In the `modules/api/src/test/functional` directory are a few important files for you to be familiar with: In the `modules/api/src/test/functional` directory are a few important files for you to be familiar with:
* `vinyl_python.py` - this provides the interface to the VinylDNS API. It handles signing the request for you, as well - `vinyl_python.py` - this provides the interface to the VinylDNS API. It handles signing the request for you, as well
as building and executing the requests, and giving you back valid responses. For all new API endpoints, there should as building and executing the requests, and giving you back valid responses. For all new API endpoints, there should
be a corresponding function in the vinyl_client be a corresponding function in the vinyl_client
* `utils.py` - provides general use functions that can be used anywhere in your tests. Feel free to contribute new - `utils.py` - provides general use functions that can be used anywhere in your tests. Feel free to contribute new
functions here when you see repetition in the code functions here when you see repetition in the code
In the `modules/api/src/test/functional/tests` directory, we have directories / modules for different areas of the In the `modules/api/src/test/functional/tests` directory, we have directories / modules for different areas of the
application. application.
* `batch` - for managing batch updates - `batch` - for managing batch updates
* `internal` - for internal endpoints (not intended for public consumption) - `internal` - for internal endpoints (not intended for public consumption)
* `membership` - for managing groups and users - `membership` - for managing groups and users
* `recordsets` - for managing record sets - `recordsets` - for managing record sets
* `zones` - for managing zones - `zones` - for managing zones
##### Functional Test Context ##### Functional Test Context

6
MAINTAINERS.md
View File

@ -27,12 +27,6 @@ The offline root key and repository keys are managed by the core maintainer team
* api key: used to sign tagged images in vinyldns/api * api key: used to sign tagged images in vinyldns/api
* portal key: used to sign tagged images in vinyldns/portal * portal key: used to sign tagged images in vinyldns/portal
These keys are named in a <hash>.key format, e.g. 5526ecd15bd413e08718e66c440d17a28968d5cd2922b59a17510da802ca6572.key,
do not change the names of the keys.
Docker expects these keys to be saved in `~/.docker/trust/private`. Each key is encrypted with a passphrase, that you
must have available when pushing an image.
## Release Process ## Release Process
The release process is automated by GitHub Actions. The release process is automated by GitHub Actions.

29
README.md
View File

@ -1,10 +1,9 @@
![Build](https://github.com/vinyldns/vinyldns/workflows/Continuous%20Integration/badge.svg) ![Build](https://github.com/vinyldns/vinyldns/workflows/Continuous%20Integration/badge.svg)
[![CodeCov ](https://codecov.io/gh/vinyldns/vinyldns/branch/master/graph/badge.svg)](https://codecov.io/gh/vinyldns/vinyldns)
[![License](https://img.shields.io/github/license/vinyldns/vinyldns)](https://github.com/vinyldns/vinyldns/blob/master/LICENSE) [![License](https://img.shields.io/github/license/vinyldns/vinyldns)](https://github.com/vinyldns/vinyldns/blob/master/LICENSE)
[![conduct](https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg)](https://github.com/vinyldns/vinyldns/blob/master/CODE_OF_CONDUCT.md) [![conduct](https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg)](https://github.com/vinyldns/vinyldns/blob/master/CODE_OF_CONDUCT.md)
<p align="left"> <p align="left">
<a href="http://www.vinyldns.io/"> <a href="https://www.vinyldns.io/">
<img <img
alt="VinylDNS" alt="VinylDNS"
src="img/vinyldns_optimized.svg" src="img/vinyldns_optimized.svg"
@ -38,6 +37,9 @@ Integration is simple with first-class language support including:
## Table of Contents ## Table of Contents
- [Quickstart](#quickstart) - [Quickstart](#quickstart)
- [Things to Try in the Portal](#things-to-try-in-the-portal)
- [Verifying Your Changes](#verifying-your-changes)
- [Other things to note](#other-things-to-note)
- [Code of Conduct](#code-of-conduct) - [Code of Conduct](#code-of-conduct)
- [Developer Guide](#developer-guide) - [Developer Guide](#developer-guide)
- [Contributing](#contributing) - [Contributing](#contributing)
@ -56,29 +58,32 @@ VinylDNS on your machine with docker:
1. Navigate to repo: `cd vinyldns` 1. Navigate to repo: `cd vinyldns`
1. Run `./quickstart/quickstart-vinyldns.sh`. This will start up the api at `localhost:9000` and the portal 1. Run `./quickstart/quickstart-vinyldns.sh`. This will start up the api at `localhost:9000` and the portal
at `localhost:9001` at `localhost:9001`
1. See [Developer Guide](DEVELOPER_GUIDE.md#loading-test-data) for how to load a test DNS zone 1. See [Things to Try in the Portal](#things-to-try-in-the-portal) for getting familiar with the Portal
1. To stop the local setup, run `./utils/clean-vinyldns-containers.sh`. 1. To stop the local setup, run `./utils/clean-vinyldns-containers.sh`.
There exist several clients at <https://github.com/vinyldns> that can be used to make API requests, using the There exist several clients at <https://github.com/vinyldns> that can be used to make API requests, using the
endpoint `http://localhost:9000` endpoint `http://localhost:9000`
## Things to try in the portal ## Things to Try in the Portal
1. View the portal at <http://localhost:9001> in a web browser 1. View the portal at <http://localhost:9001> in a web browser
2. Login with the credentials `professor` and `professor` 2. Login with the credentials `professor` and `professor`
3. Navigate to the `groups` tab: <http://localhost:9001/groups> 3. Navigate to the `groups` tab: <http://localhost:9001/groups>
4. Click on the **New Group** button and create a new group, the group id is the uuid in the url after you view the 4. Click on the **New Group** button and create a new group, the group id is the uuid in the url after you view the
group group
5. View zones you connected to in the `zones` tab: <http://localhost:9001/zones>. For a quick test, create a new zone 5. Connect a zone by going to the `zones` tab: <http://localhost:9001/zones>.
named `ok` with an email of `test@test.com` and choose a group you created from the previous step. (Note, 1. Click the `-> Connect` button
see [Developer Guide](DEVELOPER_GUIDE.md#loading-test-data) for creating a zone) 2. For `Zone Name` enter `ok` with an email of `test@test.com`
6. You will see that some records are preloaded in the zoned already, this is because these records are preloaded in the 3. For `Admin Group`, choose a group you created from the previous step
4. Leave everything else as-is and click the `Connect` button at the bottom of the form
6. A new zone `ok` should appear in your `My Zones` tab _(you may need to refresh your browser)_
7. You will see that some records are preloaded in the zone already, this is because these records are preloaded in the
local docker DNS server and VinylDNS automatically syncs records with the backend DNS server upon zone connection local docker DNS server and VinylDNS automatically syncs records with the backend DNS server upon zone connection
7. From here, you can create DNS record sets in the **Manage Records** tab, and manage zone settings and ***ACL rules*** 8. From here, you can create DNS record sets in the **Manage Records** tab, and manage zone settings and ***ACL rules***
in the **Manage Zone** tab in the **Manage Zone** tab
8. To try creating a DNS record, click on the **Create Record Set** button under 9. To try creating a DNS record, click on the **Create Record Set** button under
Records, `Record Type = A, Record Name = my-test-a, TTL = 300, IP Addressess = 1.1.1.1` Records, `Record Type = A, Record Name = my-test-a, TTL = 300, IP Addressess = 1.1.1.1`
9. Click on the **Refresh** button under Records, you should see your new record created 10. Click on the **Refresh** button under Records, you should see your new record created
### Verifying Your Changes ### Verifying Your Changes
@ -96,7 +101,7 @@ This tells `dig` to use `127.0.0.1` as the resolver on port `19001`. The `+short
verbose. Finally, the record we're looking up is `my-test-a.ok`. You can see the returned output of `1.1.1.1` matches verbose. Finally, the record we're looking up is `my-test-a.ok`. You can see the returned output of `1.1.1.1` matches
the record data we entered. the record data we entered.
## Other things to note ### Other things to note
1. Upon connecting to a zone for the first time, a zone sync is executed to provide VinylDNS a copy of the records in 1. Upon connecting to a zone for the first time, a zone sync is executed to provide VinylDNS a copy of the records in
the zone the zone

51
SYSTEM_DESIGN.md
View File

@ -1,56 +1,57 @@
# System Design # System Design
## Table of Contents ## Table of Contents
- [Components](#components) - [Components](#components)
- [Process Flow](#process-flow) - [Process Flow](#process-flow)
- [Integration](#integration) - [Integration](#integration)
## Components ## Components
The following diagram illustrates the major components in the VinylDNS ecosystem and the external systems they interact with. The following diagram illustrates the major components in the VinylDNS ecosystem and the external systems they interact
with.
![VinylDNS Architecture Diagram](img/VinylDNS_overview.png) ![VinylDNS Architecture Diagram](img/vinyldns_overview.png)
* API - RESTful endpoints to allow interaction with VinylDNS | Component | Description |
|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
* Database - stores information that the VinylDNS application needs | Portal | Web user interface to interact with the VinylDNS API |
| API | RESTful endpoints to allow interaction with VinylDNS |
* DNS servers - communicates DNS changes and resolves DNS records | API Worker Nodes | These are API components with `processing-disabled` set to `false` (see [documentation](https://www.vinyldns.io/operator/config-api.html#processing-disabled)) |
| Message queue | Queue for DNS commands to enable flow control to the DNS backends (see [documentation](https://www.vinyldns.io/operator/pre.html#message-queues)) |
* Message queue - temporarily stores DNS requests for processing | Database | Stores information about users, membership, and DNS records |
| DNS Backend(s) | The DNS backend servers which VinylDNS will query and update. |
* LDAP Service - application protocol used to authenticate user access to the VinylDNS portal | LDAP Service | The optional LDAP service that VinylDNS can be configured to communicate with (see [documentation](https://www.vinyldns.io/operator/setup-ldap.html#setup-ldap)) |
* Portal - graphical user interface to interact with the VinylDNS API
* Tooling - external libraries and utilities used to interact with the VinylDNS API
## Process Flow ## Process Flow
1. LDAP service authenticates user credentials and grants access to the portal. 1. LDAP service authenticates user credentials and grants access to the portal.
1. If the user is accessing the portal for the first time, VinylDNS credentials are generated and stored. 1. If the user is accessing the portal for the first time, VinylDNS credentials are generated and stored.
1. User navigates portal or uses integration tooling to generate a signed API request. 1. User navigates portal or uses integration tooling to generate a signed API request.
1. When the API receives a request, it loads the credentials for the calling user from the database and validates the request signature to ensure that the request was not modified in transit. 1. When the API receives a request, it loads the credentials for the calling user from the database and validates the
request signature to ensure that the request was not modified in transit.
1. The request is then validated to ensure that: 1. The request is then validated to ensure that:
- the request data is correct - the request data is correct
- the request passes all validation checks - the request passes all validation checks
- the user has access to make the change - the user has access to make the change
1. Assuming the request is in good order, the request is put on a message queue for handling. 1. Assuming the request is in good order, the request is put on a message queue for handling.
1. One of the VinylDNS API server instances pulls the message from the queue for processing. For record changes, a DDNS message is issued to the DNS backend server. 1. One of the VinylDNS API server instances pulls the message from the queue for processing. For record changes, a DDNS
1. When the message completes processing, it is removed from the message queue. The changes are applied to the VinylDNS database along with an audit record for the request. message is issued to the DNS backend server.
1. When the message completes processing, it is removed from the message queue. The changes are applied to the VinylDNS
database along with an audit record for the request.
## Integration ## Integration
Integrating with VinylDNS is simple since each API endpoint is effectively a distinct DNS operation (eg. create record, update record, delete record, etc.). The only requirement for sending a request is generating the correct AWS SIG4 signature without content length and providing the corresponding HTTP headers so that VinylDNS can verify it. See [API Authentication](https://www.vinyldns.io/api/auth-mechanism.html) for more details. Integrating with VinylDNS is simple since each API endpoint is effectively a distinct DNS operation (eg. create record,
update record, delete record, etc.). The only requirement for sending a request is generating the correct AWS SIG4
signature without content length and providing the corresponding HTTP headers so that VinylDNS can verify it.
See [API Authentication](https://www.vinyldns.io/api/auth-mechanism.html) for more details.
The current tooling available to perform VinylDNS API requests include: The current tooling available to perform VinylDNS API requests include:
* [go-vinyldns](https://github.com/vinyldns/go-vinyldns) - Golang client package * [go-vinyldns](https://github.com/vinyldns/go-vinyldns) - Golang client package
* [terraform-provider-vinyldns](https://github.com/vinyldns/terraform-provider-vinyldns) - A [Terraform](https://terraform.io/) provider for VinylDNS
* [vinyldns-cli](https://github.com/vinyldns/vinyldns-cli) - command line utility written in Golang * [vinyldns-cli](https://github.com/vinyldns/vinyldns-cli) - Command line utility written in Golang
* [vinyldns-java](https://github.com/vinyldns/vinyldns-java) - Java client * [vinyldns-java](https://github.com/vinyldns/vinyldns-java) - Java client
* [vinyldns-js](https://github.com/vinyldns/vinyldns-js) - JavaScript client
* [vinyldns-python](https://github.com/vinyldns/vinyldns-python) - Python client library * [vinyldns-python](https://github.com/vinyldns/vinyldns-python) - Python client library
* [vinyldns-ruby](https://github.com/vinyldns/vinyldns-ruby) - Ruby gem

4
build.sbt
View File

@ -5,6 +5,7 @@ import org.scalafmt.sbt.ScalafmtPlugin._
import scoverage.ScoverageKeys.{coverageFailOnMinimum, coverageMinimum} import scoverage.ScoverageKeys.{coverageFailOnMinimum, coverageMinimum}
import scala.language.postfixOps import scala.language.postfixOps
import scala.sys.env
import scala.util.Try import scala.util.Try
lazy val IntegrationTest = config("it").extend(Test) lazy val IntegrationTest = config("it").extend(Test)
@ -37,7 +38,7 @@ lazy val sharedSettings = Seq(
// coverage options // coverage options
coverageMinimum := 85, coverageMinimum := 85,
coverageFailOnMinimum := true, coverageFailOnMinimum := true,
coverageHighlighting := true coverageHighlighting := true,
) )
lazy val testSettings = Seq( lazy val testSettings = Seq(
@ -287,7 +288,6 @@ lazy val docs = (project in file("modules/docs"))
.settings(docSettings) .settings(docSettings)
def getPropertyFlagOrDefault(name: String, value: Boolean): Boolean = def getPropertyFlagOrDefault(name: String, value: Boolean): Boolean =
sys.props.get(name).flatMap(propValue => Try(propValue.toBoolean).toOption).getOrElse(value) sys.props.get(name).flatMap(propValue => Try(propValue.toBoolean).toOption).getOrElse(value)

2
build/docker/.env
View File

@ -7,7 +7,7 @@ TEST_LOGIN=false
# API Settings # API Settings
REST_PORT=9000 REST_PORT=9000
SQS_ENDPOINT=http://vinyldns-integration:19003 SQS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003 SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
MYSQL_ENDPOINT=vinyldns-integration:19002 MYSQL_ENDPOINT=vinyldns-integration:19002
DEFAULT_DNS_ADDRESS=vinyldns-integration:19001 DEFAULT_DNS_ADDRESS=vinyldns-integration:19001

8
build/docker/api/application.conf
View File

@ -49,7 +49,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME} key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET} key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1" primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS} primary-server = ${?DEFAULT_DNS_ADDRESS}
} }
transfer-connection = { transfer-connection = {
@ -58,7 +58,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME} key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET} key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1" primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS} primary-server = ${?DEFAULT_DNS_ADDRESS}
}, },
tsig-usage = "always" tsig-usage = "always"
@ -71,7 +71,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME} key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET} key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1" primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS} primary-server = ${?DEFAULT_DNS_ADDRESS}
} }
transfer-connection = { transfer-connection = {
@ -80,7 +80,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME} key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET} key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1" primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS} primary-server = ${?DEFAULT_DNS_ADDRESS}
}, },
tsig-usage = "always" tsig-usage = "always"

2
build/sbt.sh
View File

@ -4,4 +4,4 @@ set -euo pipefail
DIR=$(cd -P -- "$(dirname -- "$0")" && pwd -P) DIR=$(cd -P -- "$(dirname -- "$0")" && pwd -P)
cd "$DIR/../test/api/integration" cd "$DIR/../test/api/integration"
make build DOCKER_PARAMS="--build-arg SKIP_API_BUILD=true" && make run-local WITH_ARGS="sbt" DOCKER_PARAMS="-e RUN_SERVICES=none" make build DOCKER_PARAMS="--build-arg SKIP_API_BUILD=true" && make run-local WITH_ARGS="sbt" DOCKER_PARAMS="-e RUN_SERVICES=none --env-file \"$DIR/../test/api/integration/.env.integration\""

BIN
img/VinylDNS_overview.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 55 KiB

BIN
img/vinyldns_overview.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 91 KiB

343
modules/api/src/it/resources/application.conf
View File

@ -1,61 +1,266 @@
akka { vinyldns {
loglevel = "OFF" base-version = "0.0.0-local-dev"
log-dead-letters-during-shutdown = off version = ${vinyldns.base-version} # default to the base version if not overridden
log-dead-letters = 0 version = ${?VINYLDNS_VERSION} # override the base version via env var
logger-startup-timeout = 60s
# How often to any particular zone can be synchronized in milliseconds
sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
} }
vinyldns {
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
rest {
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
approved-name-servers = [ approved-name-servers = [
"172.17.42.1." "172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
] ]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql { mysql {
settings { settings {
# see https://github.com/brettwooldridge/HikariCP # JDBC Settings, these are all values in scalikejdbc-config, not our own
connection-timeout-millis = 1000 # these must be overridden to use MYSQL for production use
idle-timeout = 10000 # assumes a docker or mysql instance running locally
max-lifetime = 600000 name = "vinyldns"
maximum-pool-size = 5 name = ${?DATABASE_NAME}
minimum-idle = 1 driver = "org.h2.Driver"
my-sql-properties = { driver = ${?JDBC_DRIVER}
cachePrepStmts=true migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
prepStmtCacheSize=250 migration-url = ${?JDBC_MIGRATION_URL}
prepStmtCacheSqlLimit=2048 url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
rewriteBatchedStatements=true url = ${?JDBC_URL}
} user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
} }
# TODO: Remove the need for these useless configuration blocks
repositories { repositories {
zone { zone {
# no additional settings for now
} }
batch-change { batch-change {
# no additional settings for now
} }
user { user {
} }
record-set { record-set {
}
group {
}
membership {
}
group-change {
} }
zone-change { zone-change {
} }
record-change { record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
} }
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
} }
# FQDNs / IPs that cannot be modified via VinylDNS # FQDNs / IPs that cannot be modified via VinylDNS
@ -66,31 +271,65 @@ vinyldns {
"high-value-domain.*" # for testing "high-value-domain.*" # for testing
] ]
ip-list = [ ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252", "192.0.2.252",
"192.0.2.253", "192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff", "fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0" "fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
] ]
} }
# types of unowned records that users can access in shared zones global-acl-rules = [
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"] {
group-ids: ["global-acl-group-id"],
crypto { fqdn-regex-list: [".*shared[0-9]{1}."]
type = "vinyldns.core.crypto.NoOpCrypto" },
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
} }
email.settings.smtp { akka {
port = 19025 loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
} }
} }
# Global settings akka.http {
scalikejdbc.global.loggingSQLAndTime.enabled=true server {
scalikejdbc.global.loggingSQLAndTime.logLevel=error # The time period within which the TCP binding process must be completed.
scalikejdbc.global.loggingSQLAndTime.warningEnabled=true # Set to `infinite` to disable.
scalikejdbc.global.loggingSQLAndTime.warningThresholdMillis=1000 bind-timeout = 5s
scalikejdbc.global.loggingSQLAndTime.warningLogLevel=warn
scalikejdbc.global.loggingSQLAndTime.singleLineMode=false # Show verbose error messages back to the client
scalikejdbc.global.loggingSQLAndTime.printUnprocessedStackTrace=false verbose-error-messages = on
scalikejdbc.global.loggingSQLAndTime.stackTraceDepth=10 }
parsing {
# Spray doesn't like the AWS4 headers
illegal-header-warnings = on
}
}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

7
modules/api/src/it/scala/vinyldns/api/backend/dns/DnsBackendIntegrationSpec.scala
View File

@ -34,15 +34,16 @@ import vinyldns.core.domain.zone.{Algorithm, Zone, ZoneConnection}
class DnsBackendIntegrationSpec extends AnyWordSpec with Matchers { class DnsBackendIntegrationSpec extends AnyWordSpec with Matchers {
private val testConnection = ZoneConnection( private val testConnection = ZoneConnection(
"test", "vinyldns.",
"test", "vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==", "nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001", sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001"),
Algorithm.HMAC_MD5 Algorithm.HMAC_MD5
) )
"DNSBackend" should { "DNSBackend" should {
"connect to a zone without a tsig key for transfer or update" in { "connect to a zone without a tsig key for transfer or update" in {
val config = val config =
DnsBackendConfig( DnsBackendConfig(
"test", "test",

6
modules/api/src/it/scala/vinyldns/api/domain/zone/ZoneViewLoaderIntegrationSpec.scala
View File

@ -51,7 +51,7 @@ class ZoneViewLoaderIntegrationSpec extends AnyWordSpec with Matchers {
"vinyldns.", "vinyldns.",
"vinyldns.", "vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==", "nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001" sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001")
) )
), ),
transferConnection = transferConnection =
@ -84,7 +84,7 @@ class ZoneViewLoaderIntegrationSpec extends AnyWordSpec with Matchers {
"vinyldns.", "vinyldns.",
"vinyldns.", "vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==", "nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001" sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001")
) )
), ),
transferConnection = Some( transferConnection = Some(
@ -92,7 +92,7 @@ class ZoneViewLoaderIntegrationSpec extends AnyWordSpec with Matchers {
"vinyldns.", "vinyldns.",
"vinyldns.", "vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==", "nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001" sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001")
) )
) )
) )

11
modules/api/src/it/scala/vinyldns/api/notifier/sns/SnsNotifierIntegrationSpec.scala
View File

@ -16,7 +16,7 @@
package vinyldns.api.notifier.sns package vinyldns.api.notifier.sns
import cats.effect.IO import cats.effect.{IO, Timer}
import com.amazonaws.auth.{AWSStaticCredentialsProvider, BasicAWSCredentials} import com.amazonaws.auth.{AWSStaticCredentialsProvider, BasicAWSCredentials}
import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration
import com.amazonaws.services.sns.AmazonSNSClientBuilder import com.amazonaws.services.sns.AmazonSNSClientBuilder
@ -26,6 +26,7 @@ import org.joda.time.DateTime
import org.json4s.DefaultFormats import org.json4s.DefaultFormats
import org.json4s.jackson.JsonMethods._ import org.json4s.jackson.JsonMethods._
import org.scalatest.matchers.should.Matchers import org.scalatest.matchers.should.Matchers
import org.scalatest.time.SpanSugar.convertIntToGrainOfTime
import org.scalatest.wordspec.AnyWordSpecLike import org.scalatest.wordspec.AnyWordSpecLike
import vinyldns.api.MySqlApiIntegrationSpec import vinyldns.api.MySqlApiIntegrationSpec
import vinyldns.core.TestMembershipData._ import vinyldns.core.TestMembershipData._
@ -34,6 +35,8 @@ import vinyldns.core.domain.record.{AData, RecordType}
import vinyldns.core.notifier._ import vinyldns.core.notifier._
import vinyldns.mysql.MySqlIntegrationSpec import vinyldns.mysql.MySqlIntegrationSpec
import scala.concurrent.ExecutionContext
class SnsNotifierIntegrationSpec class SnsNotifierIntegrationSpec
extends MySqlApiIntegrationSpec extends MySqlApiIntegrationSpec
with MySqlIntegrationSpec with MySqlIntegrationSpec
@ -43,7 +46,7 @@ class SnsNotifierIntegrationSpec
import vinyldns.api.domain.DomainValidations._ import vinyldns.api.domain.DomainValidations._
implicit val formats = DefaultFormats implicit val formats = DefaultFormats
implicit val timer: Timer[IO] = IO.timer(ExecutionContext.global)
val snsConfig: Config = ConfigFactory.load().getConfig("vinyldns.sns.settings") val snsConfig: Config = ConfigFactory.load().getConfig("vinyldns.sns.settings")
"Sns Notifier" should { "Sns Notifier" should {
@ -91,7 +94,7 @@ class SnsNotifierIntegrationSpec
val sqs = AmazonSQSClientBuilder val sqs = AmazonSQSClientBuilder
.standard() .standard()
.withEndpointConfiguration( .withEndpointConfiguration(
new EndpointConfiguration("http://127.0.0.1:19003", "us-east-1") new EndpointConfiguration(sys.env.getOrElse("SNS_SERVICE_ENDPOINT","http://vinyldns-integration:19003"), "us-east-1")
) )
.withCredentials(credentialsProvider) .withCredentials(credentialsProvider)
.build() .build()
@ -103,7 +106,7 @@ class SnsNotifierIntegrationSpec
notifier <- new SnsNotifierProvider() notifier <- new SnsNotifierProvider()
.load(NotifierConfig("", snsConfig), userRepository) .load(NotifierConfig("", snsConfig), userRepository)
_ <- notifier.notify(Notification(batchChange)) _ <- notifier.notify(Notification(batchChange))
_ <- IO { Thread.sleep(100) } _ <- IO.sleep(1.seconds)
messages <- IO { sqs.receiveMessage(queueUrl).getMessages } messages <- IO { sqs.receiveMessage(queueUrl).getMessages }
_ <- IO { _ <- IO {
sns.deleteTopic(topic) sns.deleteTopic(topic)

3
modules/api/src/it/scala/vinyldns/api/route53/Route53ApiIntegrationSpec.scala
View File

@ -49,6 +49,7 @@ class Route53ApiIntegrationSpec
with MySqlApiIntegrationSpec { with MySqlApiIntegrationSpec {
private val testZone = Zone("example.com.", "test@test.com", backendId = Some("test")) private val testZone = Zone("example.com.", "test@test.com", backendId = Some("test"))
private def testConnection: Route53Backend = private def testConnection: Route53Backend =
Route53Backend Route53Backend
.load( .load(
@ -56,7 +57,7 @@ class Route53ApiIntegrationSpec
"test", "test",
Some("access"), Some("access"),
Some("secret"), Some("secret"),
"http://127.0.0.1:19003", sys.env.getOrElse("R53_SERVICE_ENDPOINT", "http://localhost:19003"),
"us-east-1" "us-east-1"
) )
) )

461
modules/api/src/main/resources/application.conf
View File

@ -1,15 +1,317 @@
################################################################################################################ vinyldns {
# This configuration is used primarily when running re-start or starting Vinyll locally. The configuration base-version = "0.0.0-local-dev"
# presumes a stand-alone Vinyll server with no backend services. version = ${vinyldns.base-version} # default to the base version if not overridden
################################################################################################################ version = ${?VINYLDNS_VERSION} # override the base version via env var
akka {
loglevel = "ERROR"
# The following settings are required to have Akka logging output to SLF4J and logback; without # How often to any particular zone can be synchronized in milliseconds
# these, akka will output to STDOUT sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
}
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
rest {
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
approved-name-servers = [
"172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql {
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
}
batch-change {
}
user {
}
record-set {
}
zone-change {
}
record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252",
"192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
]
}
global-acl-rules = [
{
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared[0-9]{1}."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
}
akka {
loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"] loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter" logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
}
} }
akka.http { akka.http {
@ -28,143 +330,6 @@ akka.http {
} }
} }
vinyldns { # You can provide configuration overrides via local.conf if you don't want to replace everything in
queue { # this configuration file
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider" include "local.conf"
messages-per-poll = 10
polling-interval = 250.millis
max-retries = 100 # Max retries for message on queue; currently only applies to MySqlMessageQueue
settings {
# AWS access key and secret.
access-key = "x"
secret-key = "x"
}
}
data-stores = ["mysql"]
mysql {
settings {
# see https://github.com/brettwooldridge/HikariCP
connection-timeout-millis = 1000
idle-timeout = 10000
max-lifetime = 600000
maximum-pool-size = 5
minimum-idle = 1
register-mbeans = true
}
repositories {
zone {
# no additional settings for now
}
batch-change {
# no additional settings for now
}
user {
}
record-set {
}
group {
}
membership {
}
group-change {
}
zone-change {
}
record-change {
}
}
}
sync-delay = 10000 # 10 second delay for resyncing zone
batch-change-limit = 1000 # Max change limit per batch request
# this key is used in order to encrypt/decrypt DNS TSIG keys. We use this dummy one for test purposes, this
# should be overridden with a real value that is hidden for production deployment
crypto {
type = "vinyldns.core.crypto.JavaCrypto"
secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.2.254",
"192.0.2.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
]
}
# types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
backends = [
{
id = "func-test-backend"
zone-connection {
name = "vinyldns."
key-name = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "127.0.0.1:19001"
}
transfer-connection {
name = "vinyldns."
key-name = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "127.0.0.1:19001"
}
}
]
multi-record-batch-change-enabled = true
# feature flag for manual batch review
manual-batch-review-enabled = true
scheduled-changes-enabled = true
global-acl-rules = [
{
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok."]
}
]
}

8
modules/api/src/main/resources/reference.conf
View File

@ -148,16 +148,24 @@ vinyldns {
defaultZoneConnection { defaultZoneConnection {
name = "vinyldns." name = "vinyldns."
name= ${?DEFAULT_DNS_KEY_NAME}
keyName = "vinyldns." keyName = "vinyldns."
keyName= ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primaryServer = "127.0.0.1:19001" primaryServer = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
} }
defaultTransferConnection { defaultTransferConnection {
name = "vinyldns." name = "vinyldns."
name= ${?DEFAULT_DNS_KEY_NAME}
keyName = "vinyldns." keyName = "vinyldns."
keyName= ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primaryServer = "127.0.0.1:19001" primaryServer = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
} }
batch-change-limit = 1000 batch-change-limit = 1000

12
modules/api/src/main/resources/vinyldns-ascii.txt
View File

@ -1,6 +1,6 @@
.__ .__ .___ _ ___ ______ _ _______
___ _|__| ____ ___.__.| | __| _/____ ______ | | / (_)___ __ __/ / __ \/ | / / ___/
\ \/ / |/ < | || | / __ |/ \ / ___/ | | / / / __ \/ / / / / / / / |/ /\__ \
\ /| | | \___ || |__/ /_/ | | \\___ \ | |/ / / / / / /_/ / / /_/ / /| /___/ /
\_/ |__|___| / ____||____/\____ |___| /____ > |___/_/_/ /_/\__, /_/_____/_/ |_//____/
\/\/ \/ \/ \/ /____/

381
modules/api/src/test/resources/application.conf
View File

@ -1,30 +1,227 @@
akka { vinyldns {
loglevel = "OFF" base-version = "0.0.0-local-dev"
loggers = ["akka.testkit.TestEventListener"] version = ${vinyldns.base-version} # default to the base version if not overridden
log-dead-letters-during-shutdown = off version = ${?VINYLDNS_VERSION} # override the base version via env var
log-dead-letters = 0
test.timefactor = 5 # How often to any particular zone can be synchronized in milliseconds
logger-startup-timeout = 30s sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
}
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
notifiers = ["test-notifier"]
test-notifier {
class-name = "someclass"
settings {
value = "test"
}
} }
vinyldns {
active-node-count = 3
sync-delay = 10000 # 10 second delay for resyncing zone
color = "blue"
rest { rest {
host = "127.0.0.1" host = "0.0.0.0"
port = 9000 port = 9000
port=${?API_SERVICE_PORT}
} }
# this key is used in order to encrypt/decrypt DNS TSIG keys. We use this dummy one for test purposes, this
# should be overridden with a real value that is hidden for production deployment
crypto {
type = "vinyldns.core.crypto.JavaCrypto"
secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
}
approved-name-servers = [ approved-name-servers = [
"some.test.ns." "172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
] ]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql {
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
}
batch-change {
}
user {
}
record-set {
}
zone-change {
}
record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS # FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR # regex-list used for all record types except PTR
# ip-list used exclusively for PTR records # ip-list used exclusively for PTR records
@ -33,6 +230,7 @@ vinyldns {
"high-value-domain.*" # for testing "high-value-domain.*" # for testing
] ]
ip-list = [ ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252", "192.0.2.252",
"192.0.2.253", "192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff", "fd69:27cc:fe91:0:0:0:0:ffff",
@ -48,80 +246,99 @@ vinyldns {
"needs-review.*" "needs-review.*"
] ]
ip-list = [ ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254", "192.0.2.254",
"192.0.2.255", "192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1", "fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2" "fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
] ]
zone-name-list = [ zone-name-list = [
"zone.needs.review." "zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
] ]
} }
# types of unowned records that users can access in shared zones # FQDNs / IPs that cannot be modified via VinylDNS
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"] # regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
# used for testing only high-value-domains = {
string-list-test = ["test"] regex-list = [
"high-value-domain.*" # for testing
mysql.repositories { ]
zone { ip-list = [
# no additional settings for now # using reverse zones in the vinyldns/bind9 docker image for testing
} "192.0.1.252",
batch-change { "192.0.1.253",
# no additional settings for now "192.0.2.252",
} "192.0.2.253",
user { "192.0.3.252",
"192.0.3.253",
} "192.0.4.252",
record-set { "192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
} "fd69:27cc:fe91:0:0:0:ffff:0",
group { "fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
} "fd69:27cc:fe93:0:0:0:0:ffff",
membership { "fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
} "fd69:27cc:fe94:0:0:0:ffff:0"
group-change { ]
}
zone-change {
}
record-change {
}
} }
notifiers = ["test-notifier"] global-acl-rules = [
test-notifier {
class-name = "someclass"
settings {
value = "test"
}
}
backends = [
{ {
id = "test" group-ids: ["global-acl-group-id"],
zone-connection { fqdn-regex-list: [".*shared[0-9]{1}."]
name = "zoneconn." },
key-name = "vinyldns." {
key = "test-key" group-ids: ["another-global-acl-group"],
primary-server = "127.0.0.1:19001" fqdn-regex-list: [".*ok[0-9]{1}."]
}
transfer-connection {
name = "transferconn."
key-name = "vinyldns."
key = "test-key"
primary-server = "127.0.0.1:19001"
}
} }
] ]
manual-batch-review-enabled = true
scheduled-changes-enabled = true
} }
akka {
loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
}
}
akka.http {
server {
# The time period within which the TCP binding process must be completed.
# Set to `infinite` to disable.
bind-timeout = 5s
# Show verbose error messages back to the client
verbose-error-messages = on
}
parsing {
# Spray doesn't like the AWS4 headers
illegal-header-warnings = on
}
}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

29
modules/api/src/test/scala/vinyldns/api/config/VinylDNSConfigSpec.scala
View File

@ -16,15 +16,18 @@
package vinyldns.api.config package vinyldns.api.config
import cats.effect.{ContextShift, IO}
import org.scalatest.BeforeAndAfterAll import org.scalatest.BeforeAndAfterAll
import org.scalatest.matchers.should.Matchers import org.scalatest.matchers.should.Matchers
import org.scalatest.wordspec.AnyWordSpec import org.scalatest.wordspec.AnyWordSpec
import vinyldns.api.backend.dns.DnsBackendProviderConfig
import vinyldns.core.domain.zone.ZoneConnection import vinyldns.core.domain.zone.ZoneConnection
import vinyldns.core.repository.RepositoryName._ import vinyldns.core.repository.RepositoryName._
class VinylDNSConfigSpec extends AnyWordSpec with Matchers with BeforeAndAfterAll { class VinylDNSConfigSpec extends AnyWordSpec with Matchers with BeforeAndAfterAll {
private val underTest: VinylDNSConfig = VinylDNSConfig.load().unsafeRunSync() private val underTest: VinylDNSConfig = VinylDNSConfig.load().unsafeRunSync()
private implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global)
"VinylDNSConfig" should { "VinylDNSConfig" should {
"load the rest config" in { "load the rest config" in {
@ -63,27 +66,19 @@ class VinylDNSConfigSpec extends AnyWordSpec with Matchers with BeforeAndAfterAl
notifierConfigs.head.settings.getString("value").shouldBe("test") notifierConfigs.head.settings.getString("value").shouldBe("test")
} }
"load default keys" in {
val defaultConn =
ZoneConnection("vinyldns.", "vinyldns.", "nzisn+4G2ldMn0q1CV3vsg==", "127.0.0.1:19001")
underTest.configuredDnsConnections.defaultZoneConnection
.decrypted(underTest.crypto) shouldBe
defaultConn
underTest.configuredDnsConnections.defaultTransferConnection
.decrypted(underTest.crypto) shouldBe
defaultConn
}
"load specified backends" in { "load specified backends" in {
val zc = ZoneConnection("zoneconn.", "vinyldns.", "test-key", "127.0.0.1:19001") val zc = ZoneConnection("vinyldns.", "vinyldns.", "nzisn+4G2ldMn0q1CV3vsg==", sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001"))
val tc = zc.copy(name = "transferconn.") val tc = zc.copy()
val backends = underTest.configuredDnsConnections.dnsBackends val backends = underTest.backendConfigs.backendProviders
backends.length shouldBe 1 backends.length shouldBe 1
backends.head.id shouldBe "test" val config = DnsBackendProviderConfig.load(backends.head.settings).unsafeRunSync()
backends.head.zoneConnection.decrypted(underTest.crypto) shouldBe zc config.backends.length shouldBe 2
backends.head.transferConnection.decrypted(underTest.crypto) shouldBe tc
config.backends.head.id shouldBe "default"
config.backends.head.zoneConnection.decrypted(underTest.crypto) shouldBe zc
config.backends.head.transferConnection.get.decrypted(underTest.crypto) shouldBe tc
} }
} }
} }

381
modules/api/src/universal/conf/application.conf
View File

@ -1,124 +1,317 @@
# The default application.conf is not intended to be used in production. It assumes a docker-compose
# setup for all of the services. Provide your own application.conf on the docker mount with your
# own settings
vinyldns { vinyldns {
base-version = "0.0.0-local-dev"
# Should be provided in the start up script via an environment
base-version = "unset"
version = ${vinyldns.base-version} # default to the base version if not overridden version = ${vinyldns.base-version} # default to the base version if not overridden
version = ${?VINYLDNS_VERSION} # override the base version via env var version = ${?VINYLDNS_VERSION} # override the base version via env var
queue.settings.service-endpoint = "http://vinyldns-elasticmq:9324/" # How often to any particular zone can be synchronized in milliseconds
# host and port the server binds to. This should not be changed
rest {
host = "0.0.0.0"
port = 9000
}
# the delay between zone syncs so we are not syncing too often
sync-delay = 10000 sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# crypto settings for symmetric cryptography of secrets in the system # If we should start up polling for change requests, set this to false for the inactive cluster
# Note: for production systems secrets should not live in plain text in a file processing-disabled = false
crypto { processing-disabled = ${?PROCESSING_DISABLED}
type = "vinyldns.core.crypto.NoOpCrypto"
}
data-stores = ["mysql"] # Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# default settings point to the setup from docker compose # Types of unowned records that users can access in shared zones
mysql { shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
settings {
name = "vinyldns"
driver = "org.mariadb.jdbc.Driver"
migration-url = "jdbc:mariadb://vinyldns-mysql:3306/?user=root&password=pass"
url = "jdbc:mariadb://vinyldns-mysql:3306/vinyldns?user=root&password=pass"
user = "root"
password = "pass"
# see https://github.com/brettwooldridge/HikariCP # Batch change settings
connection-timeout-millis = 1000 batch-change-limit = 1000
max-lifetime = 600000 batch-change-limit = ${?BATCH_CHANGE_LIMIT}
maximum-pool-size = 20 manual-batch-review-enabled = true
register-mbeans = true manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
} scheduled-changes-enabled = true
repositories { scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
zone { multi-record-batch-change-enabled = true
# no additional settings for now multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
}
batch-change {
# no additional settings for now
}
user {
# no additional settings for now
}
record-set {
# no additional settings for now
}
group {
# no additional settings for now
}
membership {
# no additional settings for now
}
group-change {
# no additional settings for now
}
zone-change {
# no additional settings for now
}
record-change {
# no additional settings for now
}
}
}
# the DDNS connection information for the default dns backend # configured backend providers
defaultZoneConnection { backend {
name = "vinyldns." # Use "default" when dns backend legacy = true
keyName = "vinyldns." # otherwise, use the id of one of the connections in any of your backends
key = "nzisn+4G2ldMn0q1CV3vsg==" default-backend-id = "default"
primaryServer = "localhost:19001"
}
# the AXFR connection information for the default dns backend
defaultTransferConnection {
name = "vinyldns."
keyName = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primaryServer = "localhost:19001"
}
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [ backends = [
{ {
id = "func-test-backend" id = "default"
zone-connection { zone-connection = {
name = "vinyldns." name = "vinyldns."
key-name = "vinyldns." key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "localhost:19001" key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
} }
transfer-connection { transfer-connection = {
name = "vinyldns." name = "vinyldns."
key-name = "vinyldns." key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg==" key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "localhost:19001" key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
} }
} }
] ]
# the max number of changes in a single batch change. Change carefully as this has performance
# implications
batch-change-limit = 1000
} }
# Akka settings, these should not need to be modified unless you know akka http really well.
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
rest {
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
approved-name-servers = [
"172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql {
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
}
batch-change {
}
user {
}
record-set {
}
zone-change {
}
record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252",
"192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
]
}
global-acl-rules = [
{
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared[0-9]{1}."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
}
akka { akka {
loglevel = "INFO" loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"] loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter" logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
}
} }
akka.http { akka.http {
@ -132,7 +325,11 @@ akka.http {
} }
parsing { parsing {
# akka-http doesn't like the AWS4 headers # Spray doesn't like the AWS4 headers
illegal-header-warnings = on illegal-header-warnings = on
} }
} }
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

2
modules/docs/src/main/mdoc/operator/config-api.md
View File

@ -52,7 +52,7 @@ snippet...
access-key = ${AWS_ACCESS_KEY} access-key = ${AWS_ACCESS_KEY}
secret-key = ${AWS_SECRET_ACCESS_KEY} secret-key = ${AWS_SECRET_ACCESS_KEY}
signing-region = ${SQS_REGION} signing-region = ${SQS_REGION}
service-endpoint = ${SQS_ENDPOINT} service-endpoint = ${SQS_SERVICE_ENDPOINT}
queue-name = ${SQS_QUEUE_NAME} queue-name = ${SQS_QUEUE_NAME}
} }
``` ```

33
modules/docs/src/main/mdoc/operator/config-portal.md
View File

@ -251,7 +251,7 @@ portal.vinyldns.backend.url = "http://vinyldns-api:9000"
portal.test_login = false portal.test_login = false
# configuration for the users and groups store # configuration for the users and groups store
data-stores = ["dynamodb", "mysql"] data-stores = ["mysql"]
mysql { mysql {
class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider" class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider"
@ -284,37 +284,6 @@ mysql {
} }
} }
dynamodb {
class-name = "vinyldns.dynamodb.repository.DynamoDBDataStoreProvider"
settings {
key = "x"
secret = "x"
endpoint = "http://vinyldns-dynamodb:8000"
region = "us-east-1"
}
repositories {
user-change {
table-name = "userChangeTest"
provisioned-reads = 30
provisioned-writes = 20
}
}
LDAP {
user="test"
password="test"
domain="test"
searchBase = [{organization = "someDomain", domainName = "DC=test,DC=test,DC=com"}, {organization = "anotherDomain", domainName = "DC=test,DC=com"}]
context {
initialContextFactory = "com.sun.jndi.ldap.LdapCtxFactory"
securityAuthentication = "simple"
providerUrl = "ldaps://somedomain.com:9999"
}
} }
play.filters.enabled += "play.filters.csrf.CSRFFilter" play.filters.enabled += "play.filters.csrf.CSRFFilter"

10
modules/mysql/src/it/resources/application.conf
View File

@ -1,5 +1,7 @@
mysql { mysql {
class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider" class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider"
endpoint = "localhost:19002"
endpoint = ${?MYSQL_ENDPOINT}
settings { settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own # JDBC Settings, these are all values in scalikejdbc-config, not our own
@ -7,8 +9,8 @@ mysql {
# assumes a docker or mysql instance running locally # assumes a docker or mysql instance running locally
name = "vinyldns2" name = "vinyldns2"
driver = "org.mariadb.jdbc.Driver" driver = "org.mariadb.jdbc.Driver"
migration-url = "jdbc:mariadb://localhost:19002/" migration-url = "jdbc:mariadb://"${mysql.endpoint}"/"
url = "jdbc:mariadb://localhost:19002/vinyldns2" url = "jdbc:mariadb://"${mysql.endpoint}"/vinyldns2"
user = "root" user = "root"
password = "pass" password = "pass"
@ -43,8 +45,8 @@ queue {
settings = { settings = {
name = "vinyldns2" name = "vinyldns2"
driver = "org.mariadb.jdbc.Driver" driver = "org.mariadb.jdbc.Driver"
migration-url = "jdbc:mariadb://localhost:19002/?user=root&password=pass" migration-url = "jdbc:mariadb://"${mysql.endpoint}"/?user=root&password=pass"
url = "jdbc:mariadb://localhost:19002/vinyldns2?user=root&password=pass" url = "jdbc:mariadb://"${mysql.endpoint}"/vinyldns2?user=root&password=pass"
user = "root" user = "root"
password = "pass" password = "pass"

82
modules/mysql/src/main/resources/test/ddl.sql
View File

@ -6,7 +6,7 @@
-- Ex: "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'" -- Ex: "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
-- --
CREATE TABLE batch_change CREATE TABLE IF NOT EXISTS batch_change
( (
id char(36) not null primary key, id char(36) not null primary key,
user_id char(36) not null, user_id char(36) not null,
@ -22,16 +22,16 @@ CREATE TABLE batch_change
cancelled_timestamp datetime null cancelled_timestamp datetime null
); );
create index batch_change_approval_status_index CREATE INDEX IF NOT EXISTS batch_change_approval_status_index
on batch_change (approval_status); on batch_change (approval_status);
create index batch_change_user_id_created_time_index CREATE INDEX IF NOT EXISTS batch_change_user_id_created_time_index
on batch_change (user_id, created_time); on batch_change (user_id, created_time);
create index batch_change_user_id_index CREATE INDEX IF NOT EXISTS batch_change_user_id_index
on batch_change (user_id); on batch_change (user_id);
create table group_change CREATE TABLE IF NOT EXISTS group_change
( (
id char(36) not null primary key, id char(36) not null primary key,
group_id char(36) not null, group_id char(36) not null,
@ -39,10 +39,10 @@ create table group_change
data blob not null data blob not null
); );
create index group_change_group_id_index CREATE INDEX IF NOT EXISTS group_change_group_id_index
on group_change (group_id); on group_change (group_id);
create table `groups` CREATE TABLE IF NOT EXISTS `groups`
( (
id char(36) not null primary key, id char(36) not null primary key,
name varchar(256) not null, name varchar(256) not null,
@ -52,10 +52,10 @@ create table `groups`
email varchar(256) not null email varchar(256) not null
); );
create index groups_name_index CREATE INDEX IF NOT EXISTS groups_name_index
on `groups` (name); on `groups` (name);
create table membership CREATE TABLE IF NOT EXISTS membership
( (
user_id char(36) not null, user_id char(36) not null,
group_id char(36) not null, group_id char(36) not null,
@ -63,7 +63,7 @@ create table membership
primary key (user_id, group_id) primary key (user_id, group_id)
); );
create table message_queue CREATE TABLE IF NOT EXISTS message_queue
( (
id char(36) not null primary key, id char(36) not null primary key,
message_type tinyint null, message_type tinyint null,
@ -75,16 +75,16 @@ create table message_queue
attempts int default 0 not null attempts int default 0 not null
); );
create index message_queue_inflight_index CREATE INDEX IF NOT EXISTS message_queue_inflight_index
on message_queue (in_flight); on message_queue (in_flight);
create index message_queue_timeout_index CREATE INDEX IF NOT EXISTS message_queue_timeout_index
on message_queue (timeout_seconds); on message_queue (timeout_seconds);
create index message_queue_updated_index CREATE INDEX IF NOT EXISTS message_queue_updated_index
on message_queue (updated); on message_queue (updated);
create table record_change CREATE TABLE IF NOT EXISTS record_change
( (
id char(36) not null primary key, id char(36) not null primary key,
zone_id char(36) not null, zone_id char(36) not null,
@ -93,13 +93,13 @@ create table record_change
data blob not null data blob not null
); );
create index record_change_created_index CREATE INDEX IF NOT EXISTS record_change_created_index
on record_change (created); on record_change (created);
create index record_change_zone_id_index CREATE INDEX IF NOT EXISTS record_change_zone_id_index
on record_change (zone_id); on record_change (zone_id);
create table recordset CREATE TABLE IF NOT EXISTS recordset
( (
id char(36) not null primary key, id char(36) not null primary key,
zone_id char(36) not null, zone_id char(36) not null,
@ -112,16 +112,16 @@ create table recordset
unique (zone_id, name, type) unique (zone_id, name, type)
); );
create index recordset_fqdn_index CREATE INDEX IF NOT EXISTS recordset_fqdn_index
on recordset (fqdn); on recordset (fqdn);
create index recordset_owner_group_id_index CREATE INDEX IF NOT EXISTS recordset_owner_group_id_index
on recordset (owner_group_id); on recordset (owner_group_id);
create index recordset_type_index CREATE INDEX IF NOT EXISTS recordset_type_index
on recordset (type); on recordset (type);
create table single_change CREATE TABLE IF NOT EXISTS single_change
( (
id char(36) not null primary key, id char(36) not null primary key,
seq_num smallint not null, seq_num smallint not null,
@ -138,13 +138,13 @@ create table single_change
on delete cascade on delete cascade
); );
create index single_change_batch_change_id_index CREATE INDEX IF NOT EXISTS single_change_batch_change_id_index
on single_change (batch_change_id); on single_change (batch_change_id);
create index single_change_record_set_change_id_index CREATE INDEX IF NOT EXISTS single_change_record_set_change_id_index
on single_change (record_set_change_id); on single_change (record_set_change_id);
create table stats CREATE TABLE IF NOT EXISTS stats
( (
id bigint auto_increment primary key, id bigint auto_increment primary key,
name varchar(255) not null, name varchar(255) not null,
@ -152,13 +152,13 @@ create table stats
created datetime not null created datetime not null
); );
create index stats_name_created_index CREATE INDEX IF NOT EXISTS stats_name_created_index
on stats (name, created); on stats (name, created);
create index stats_name_index CREATE INDEX IF NOT EXISTS stats_name_index
on stats (name); on stats (name);
create table task CREATE TABLE IF NOT EXISTS task
( (
name varchar(255) not null primary key, name varchar(255) not null primary key,
in_flight bit not null, in_flight bit not null,
@ -166,7 +166,7 @@ create table task
updated datetime null updated datetime null
); );
create table user CREATE TABLE IF NOT EXISTS user
( (
id char(36) not null primary key, id char(36) not null primary key,
user_name varchar(256) not null, user_name varchar(256) not null,
@ -174,13 +174,13 @@ create table user
data blob not null data blob not null
); );
create index user_access_key_index CREATE INDEX IF NOT EXISTS user_access_key_index
on user (access_key); on user (access_key);
create index user_user_name_index CREATE INDEX IF NOT EXISTS user_user_name_index
on user (user_name); on user (user_name);
create table user_change CREATE TABLE IF NOT EXISTS user_change
( (
change_id char(36) not null primary key, change_id char(36) not null primary key,
user_id char(36) not null, user_id char(36) not null,
@ -188,7 +188,7 @@ create table user_change
created_timestamp bigint(13) not null created_timestamp bigint(13) not null
); );
create table zone CREATE TABLE IF NOT EXISTS zone
( (
id char(36) not null primary key, id char(36) not null primary key,
name varchar(256) not null, name varchar(256) not null,
@ -198,13 +198,13 @@ create table zone
unique (name) unique (name)
); );
create index zone_admin_group_id_index CREATE INDEX IF NOT EXISTS zone_admin_group_id_index
on zone (admin_group_id); on zone (admin_group_id);
create index zone_name_index CREATE INDEX IF NOT EXISTS zone_name_index
on zone (name); on zone (name);
create table zone_access CREATE TABLE IF NOT EXISTS zone_access
( (
accessor_id char(36) not null, accessor_id char(36) not null,
zone_id char(36) not null, zone_id char(36) not null,
@ -214,13 +214,13 @@ create table zone_access
on delete cascade on delete cascade
); );
create index zone_access_accessor_id_index CREATE INDEX IF NOT EXISTS zone_access_accessor_id_index
on zone_access (accessor_id); on zone_access (accessor_id);
create index zone_access_zone_id_index CREATE INDEX IF NOT EXISTS zone_access_zone_id_index
on zone_access (zone_id); on zone_access (zone_id);
create table zone_change CREATE TABLE IF NOT EXISTS zone_change
( (
change_id char(36) not null primary key, change_id char(36) not null primary key,
zone_id char(36) not null, zone_id char(36) not null,
@ -228,11 +228,13 @@ create table zone_change
created_timestamp bigint(13) not null created_timestamp bigint(13) not null
); );
create index zone_change_created_timestamp_index CREATE INDEX IF NOT EXISTS zone_change_created_timestamp_index
on zone_change (created_timestamp); on zone_change (created_timestamp);
create index zone_change_zone_id_index CREATE INDEX IF NOT EXISTS zone_change_zone_id_index
on zone_change (zone_id); on zone_change (zone_id);
DELETE FROM task WHERE name = 'user_sync';
INSERT IGNORE INTO task(name, in_flight, created, updated) INSERT IGNORE INTO task(name, in_flight, created, updated)
VALUES ('user_sync', 0, NOW(), NULL); VALUES ('user_sync', 0, NOW(), NULL);

13
modules/mysql/src/main/scala/vinyldns/mysql/MySqlConnector.scala
View File

@ -90,14 +90,21 @@ object MySqlConnector {
def retry[T](times: Int, delayMs: Int)(op: => T) = def retry[T](times: Int, delayMs: Int)(op: => T) =
Iterator Iterator
.range(0, times) .range(0, times)
.map(_ => Try(op)) .map(index => (index, Try(op)))
.flatMap { .flatMap(x => {
val (currentIndex, result) = x
result match {
case Success(t) => Some(t) case Success(t) => Some(t)
case Failure(_) => case Failure(e) =>
logger.warn("failed to startup database connection, retrying..") logger.warn("failed to startup database connection, retrying..")
// Hard abort if we exhaust retries
if (currentIndex >= times - 1) {
throw e
}
Thread.sleep(delayMs) Thread.sleep(delayMs)
None None
} }
})
.toSeq .toSeq
.head .head

71
modules/portal/conf/application.conf
View File

@ -1,23 +1,62 @@
http.port = 9001 LDAP {
# For OpenLDAP, this would be a full DN to the admin for LDAP / user that can see all users
user = "cn=admin,dc=planetexpress,dc=com"
crypto { # Password for the admin account
type = "vinyldns.core.crypto.JavaCrypto" password = "GoodNewsEveryone"
secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
# Keep this as an empty string for OpenLDAP
domain = ""
# This will be the name of the LDAP field that carries the user's login id (what they enter in the username in login form)
userNameAttribute = "uid"
# For organization, leave empty for this demo, the domainName is what matters, and that is the LDAP structure
# to search for users that require login
searchBase = [
{organization = "", domainName = "ou=people,dc=planetexpress,dc=com"},
]
context {
initialContextFactory = "com.sun.jndi.ldap.LdapCtxFactory"
initialContextFactory = ${?LDAP_INITIAL_CONTEXT_CLASS}
securityAuthentication = "simple"
securityAuthentication = ${?LDAP_SECURITY_AUTH}
# Note: The following assumes a purely docker setup, using container_name = vinyldns-ldap
providerUrl = "ldap://vinyldns-ldap:19004"
providerUrl = ${?LDAP_PROVIDER_URL}
} }
# This is only needed if keeping vinyldns user store in sync with ldap (to auto lock out users who left your
# company for example)
user-sync {
enabled = false
enabled = ${?USER_SYNC_ENABLED}
hours-polling-interval = 1
hours-polling-interval = ${?USER_SYNC_POLL_INTERVAL}
}
}
# Note: This MUST match the API or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
http.port = 9001
http.port = ${?PORTAL_PORT}
data-stores = ["mysql"] data-stores = ["mysql"]
data-stores = ${?DATA_STORES}
mysql { # Must be true to manage shared zones through the portal
repositories { shared-display-enabled = true
user { shared-display-enabled = ${?SHARED_ZONES_ENABLED}
}
task {
}
user-change {
}
}
}
// Local.conf has files specific to your environment, for example your own LDAP settings # You generate this yourself following https://www.playframework.com/documentation/2.7.x/ApplicationSecret
play.http.secret.key = "changeme"
play.http.secret.key = ${?PLAY_HTTP_SECRET_KEY}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf" include "local.conf"

2
modules/r53/src/it/scala/vinyldns/route53/backend/Route53IntegrationSpec.scala
View File

@ -52,7 +52,7 @@ class Route53IntegrationSpec
"test", "test",
Option("access"), Option("access"),
Option("secret"), Option("secret"),
"http://127.0.0.1:19003", sys.env.getOrElse("R53_SERVICE_ENDPOINT", "http://localhost:19003"),
"us-east-1" "us-east-1"
) )
) )

1
modules/sqs/src/it/resources/application.conf
View File

@ -8,6 +8,7 @@ sqs {
secret-key = "test" secret-key = "test"
signing-region = "us-east-1" signing-region = "us-east-1"
service-endpoint = "http://localhost:19003/" service-endpoint = "http://localhost:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
queue-name = "sqs-override-name" queue-name = "sqs-override-name"
} }
} }

37
modules/sqs/src/it/scala/vinyldns/sqs/queue/SqsMessageQueueProviderIntegrationSpec.scala
View File

@ -15,6 +15,7 @@
*/ */
package vinyldns.sqs.queue package vinyldns.sqs.queue
import com.typesafe.config.ConfigFactory import com.typesafe.config.ConfigFactory
import org.scalatest.matchers.should.Matchers import org.scalatest.matchers.should.Matchers
import org.scalatest.wordspec.AnyWordSpec import org.scalatest.wordspec.AnyWordSpec
@ -24,33 +25,36 @@ import pureconfig._
import pureconfig.generic.auto._ import pureconfig.generic.auto._
class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers { class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
val undertest = new SqsMessageQueueProvider() val underTest = new SqsMessageQueueProvider()
private val sqsEndpoint = sys.env.getOrElse("SQS_SERVICE_ENDPOINT", "http://localhost:19003")
"load" should { "load" should {
"fail if a required setting is not provided" in { "fail if a required setting is not provided" in {
val badConfig = val badConfig =
ConfigFactory.parseString(""" ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider" | class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis | polling-interval = 250.millis
| messages-per-poll = 10 | messages-per-poll = 10
| max-retries = 100 | max-retries = 100
| |
| settings { | settings {
| service-endpoint = "http://localhost:19003/" | service-endpoint = "$sqsEndpoint"
| queue-name = "queue-name" | queue-name = "queue-name"
| } | }
| """.stripMargin) | """.stripMargin)
val badSettings = ConfigSource.fromConfig(badConfig).loadOrThrow[MessageQueueConfig] val badSettings = ConfigSource.fromConfig(badConfig).loadOrThrow[MessageQueueConfig]
a[pureconfig.error.ConfigReaderException[MessageQueueConfig]] should be thrownBy undertest a[pureconfig.error.ConfigReaderException[MessageQueueConfig]] should be thrownBy underTest
.load(badSettings) .load(badSettings)
.unsafeRunSync() .unsafeRunSync()
} }
"create the queue if the queue is non-existent" in { "create the queue if the queue is non-existent" in {
val nonExistentQueueConfig = val nonExistentQueueConfig =
ConfigFactory.parseString(""" ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider" | class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis | polling-interval = 250.millis
| messages-per-poll = 10 | messages-per-poll = 10
@ -60,14 +64,14 @@ class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
| access-key = "x" | access-key = "x"
| secret-key = "x" | secret-key = "x"
| signing-region = "us-east-1" | signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/" | service-endpoint = "$sqsEndpoint"
| queue-name = "new-queue" | queue-name = "new-queue"
| } | }
| """.stripMargin) | """.stripMargin)
val messageConfig = val messageConfig =
ConfigSource.fromConfig(nonExistentQueueConfig).loadOrThrow[MessageQueueConfig] ConfigSource.fromConfig(nonExistentQueueConfig).loadOrThrow[MessageQueueConfig]
val messageQueue = undertest.load(messageConfig).unsafeRunSync() val messageQueue = underTest.load(messageConfig).unsafeRunSync()
noException should be thrownBy messageQueue noException should be thrownBy messageQueue
.asInstanceOf[SqsMessageQueue] .asInstanceOf[SqsMessageQueue]
@ -77,7 +81,8 @@ class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
"fail with InvalidQueueName if an invalid queue name is given" in { "fail with InvalidQueueName if an invalid queue name is given" in {
val invalidQueueNameConfig = val invalidQueueNameConfig =
ConfigFactory.parseString(""" ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider" | class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis | polling-interval = 250.millis
| messages-per-poll = 10 | messages-per-poll = 10
@ -87,19 +92,20 @@ class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
| access-key = "x" | access-key = "x"
| secret-key = "x" | secret-key = "x"
| signing-region = "us-east-1" | signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/" | service-endpoint = "$sqsEndpoint"
| queue-name = "bad*queue*name" | queue-name = "bad*queue*name"
| } | }
| """.stripMargin) | """.stripMargin)
val messageConfig = val messageConfig =
ConfigSource.fromConfig(invalidQueueNameConfig).loadOrThrow[MessageQueueConfig] ConfigSource.fromConfig(invalidQueueNameConfig).loadOrThrow[MessageQueueConfig]
assertThrows[InvalidQueueName](undertest.load(messageConfig).unsafeRunSync()) assertThrows[InvalidQueueName](underTest.load(messageConfig).unsafeRunSync())
} }
"fail with InvalidQueueName if a FIFO queue is specified" in { "fail with InvalidQueueName if a FIFO queue is specified" in {
val fifoQueueName = val fifoQueueName =
ConfigFactory.parseString(""" ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider" | class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis | polling-interval = 250.millis
| messages-per-poll = 10 | messages-per-poll = 10
@ -109,20 +115,21 @@ class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
| access-key = "x" | access-key = "x"
| secret-key = "x" | secret-key = "x"
| signing-region = "us-east-1" | signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/" | service-endpoint = "$sqsEndpoint"
| queue-name = "queue.fifo" | queue-name = "queue.fifo"
| } | }
| """.stripMargin) | """.stripMargin)
val messageConfig = ConfigSource.fromConfig(fifoQueueName).loadOrThrow[MessageQueueConfig] val messageConfig = ConfigSource.fromConfig(fifoQueueName).loadOrThrow[MessageQueueConfig]
assertThrows[InvalidQueueName](undertest.load(messageConfig).unsafeRunSync()) assertThrows[InvalidQueueName](underTest.load(messageConfig).unsafeRunSync())
} }
} }
"MessageQueueLoader" should { "MessageQueueLoader" should {
"invoke SQS provider properly" in { "invoke SQS provider properly" in {
val nonExistentQueueConfig = val nonExistentQueueConfig =
ConfigFactory.parseString(""" ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider" | class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis | polling-interval = 250.millis
| messages-per-poll = 10 | messages-per-poll = 10
@ -132,7 +139,7 @@ class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
| access-key = "x" | access-key = "x"
| secret-key = "x" | secret-key = "x"
| signing-region = "us-east-1" | signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/" | service-endpoint = "$sqsEndpoint"
| queue-name = "new-queue" | queue-name = "new-queue"
| } | }
| """.stripMargin) | """.stripMargin)

3
quickstart/.env
View File

@ -11,8 +11,9 @@ TEST_LOGIN=false
# API Settings # API Settings
REST_PORT=9000 REST_PORT=9000
SQS_ENDPOINT=http://vinyldns-integration:19003 SQS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003 SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
R53_SERVICE_ENDPOINT=http://vinyldns-integration:19003
MYSQL_ENDPOINT=vinyldns-integration:19002 MYSQL_ENDPOINT=vinyldns-integration:19002
DEFAULT_DNS_ADDRESS=vinyldns-integration:19001 DEFAULT_DNS_ADDRESS=vinyldns-integration:19001

2
test/api/functional/application.conf
View File

@ -85,7 +85,7 @@ vinyldns {
# Endpoint to access queue # Endpoint to access queue
service-endpoint = "http://localhost:19003/" service-endpoint = "http://localhost:19003/"
service-endpoint = ${?SQS_ENDPOINT} service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change. # Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns" queue-name = "vinyldns"

27
test/api/integration/.env.integration Executable file
View File

@ -0,0 +1,27 @@
# Overrides that can be used when dependent services are running in the vinyldns-integration host
# and 'sbt' or tests are running in another container. More importantly, when these services are
# not accessible via "localhost"
#
# Any docker container that wishes to to use these, can simply specify this file using the `--env-file`
# argument.
# General settings
VINYLDNS_API_URL=http://vinyldns-integration:9000
VINYLDNS_PORTAL_URL=http://vinyldns-integration:9001
# Portal settings
VINYLDNS_BACKEND_URL=http://vinyldns-integration:9000
LDAP_PROVIDER_URL=ldap://vinyldns-integration:19004
# API Settings
SQS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
R53_SERVICE_ENDPOINT=http://vinyldns-integration:19003
MYSQL_ENDPOINT=vinyldns-integration:19002
DEFAULT_DNS_ADDRESS=vinyldns-integration:19001
JDBC_DRIVER=org.mariadb.jdbc.Driver
JDBC_USER=root
JDBC_PASSWORD=pass
JDBC_URL=jdbc:mariadb://vinyldns-integration:19002/vinyldns?user=${JDBC_USER}&password=${JDBC_PASSWORD}
JDBC_MIGRATION_URL=jdbc:mariadb://vinyldns-integration:19002/?user=${JDBC_USER}&password=${JDBC_PASSWORD}

9
test/api/integration/Makefile
View File

@ -38,14 +38,16 @@ build:
run: run:
@set -euo pipefail @set -euo pipefail
docker network create --driver bridge vinyldns_net &> /dev/null || true
USE_TTY="" && test -t 1 && USE_TTY="-t" USE_TTY="" && test -t 1 && USE_TTY="-t"
docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) $(IMAGE_NAME) $(ARG_SEPARATOR) $(WITH_ARGS) docker run -i $${USE_TTY} --rm --network vinyldns_net $(DOCKER_PARAMS) $(IMAGE_NAME) $(ARG_SEPARATOR) $(WITH_ARGS)
run-bg: run-bg:
@set -euo pipefail @set -euo pipefail
docker stop $(IMAGE_NAME) &> /dev/null || true docker stop $(IMAGE_NAME) &> /dev/null || true
docker network create --driver bridge vinyldns_net &> /dev/null || true
USE_TTY="" && test -t 1 && USE_TTY="-t" USE_TTY="" && test -t 1 && USE_TTY="-t"
docker run -d $${USE_TTY} --name $(IMAGE_NAME) --rm $(DOCKER_PARAMS) -e RUN_SERVICES="deps-only tail-logs" -p 19001-19003:19001-19003 -p 19001:19001/udp $(IMAGE_NAME) docker run -d $${USE_TTY} --name $(IMAGE_NAME) --rm --network vinyldns_net $(DOCKER_PARAMS) -e RUN_SERVICES="deps-only tail-logs" -p 19001-19003:19001-19003 -p 19001:19001/udp $(IMAGE_NAME)
stop-bg: stop-bg:
@set -euo pipefail @set -euo pipefail
@ -53,8 +55,9 @@ stop-bg:
run-local: run-local:
@set -euo pipefail @set -euo pipefail
docker network create --driver bridge vinyldns_net &> /dev/null || true
USE_TTY="" && test -t 1 && USE_TTY="-t" USE_TTY="" && test -t 1 && USE_TTY="-t"
docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) -v "$(ROOT_DIR)/../../..:/build" $(IMAGE_NAME) -- $(WITH_ARGS) docker run -i $${USE_TTY} --rm --network vinyldns_net $(DOCKER_PARAMS) -v "$(ROOT_DIR)/../../..:/build" $(IMAGE_NAME) -- $(WITH_ARGS)
clean-containers: clean-containers:
@set -euo pipefail @set -euo pipefail

2
test/api/integration/application.conf
View File

@ -114,7 +114,7 @@ vinyldns {
# Endpoint to access queue # Endpoint to access queue
service-endpoint = "http://localhost:19003/" service-endpoint = "http://localhost:19003/"
service-endpoint = ${?SQS_ENDPOINT} service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change. # Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns" queue-name = "vinyldns"