2
0
mirror of https://github.com/VinylDNS/vinyldns synced 2025-08-22 02:02:14 +00:00

Consistency Updates

- Clean up documentation
- Update architecure diagram
- Fix discrepencies between local and docker test execution
- Fix inconsistencies in various configuration files used for tests and
execution
This commit is contained in:
Ryan Emerle 2021-12-08 14:36:00 -05:00
parent 9913b67d75
commit 85577a27f0
No known key found for this signature in database
GPG Key ID: C0D34C592AED41CE
36 changed files with 1597 additions and 722 deletions

View File

@ -3,14 +3,9 @@
This project would not be possible without the generous contributions of many people. Thank you! If you have contributed
in any way, but do not see your name here, please open a PR to add yourself (in alphabetical order by last name)!
## DNS SMEs
- Joe Crowe
- David Back
- Hong Ye
## Contributors
- David Back
- Mike Ball
- Tommy Barker
- Robert Barrimond
@ -23,6 +18,7 @@ in any way, but do not see your name here, please open a PR to add yourself (in
- Peter Cline
- Kemar Cockburn
- Luke Cori
- Joe Crowe
- Jearvon Dharrie
- Andrew Dunn
- Ryan Emerle
@ -54,3 +50,4 @@ in any way, but do not see your name here, please open a PR to add yourself (in
- Andrew Wang
- Peter Willis
- Britney Wright
- Hong Ye

View File

@ -2,10 +2,28 @@
## Table of Contents
- [Developer Requirements](#developer-requirements)
- [Developer Requirements (Local)](#developer-requirements-local)
- [Developer Requirements (Docker)](#developer-requirements-docker)
- [Project Layout](#project-layout)
* [Core](#core)
* [API](#api)
* [Portal](#portal)
* [Documentation](#documentation)
- [Running VinylDNS Locally](#running-vinyldns-locally)
* [Starting the API Server](#starting-the-api-server)
* [Starting the Portal](#starting-the-portal)
- [Testing](#testing)
* [Unit Tests](#unit-tests)
* [Integration Tests](#integration-tests)
+ [Running both](#running-both)
* [Functional Tests](#functional-tests)
+ [Running Functional Tests](#running-functional-tests)
- [API Functional Tests](#api-functional-tests)
+ [Setup](#setup)
- [Functional Test Context](#functional-test-context)
- [Partitioning](#partitioning)
- [Really Important Test Context Rules!](#really-important-test-context-rules)
- [Managing Test Zone Files](#managing-test-zone-files)
## Developer Requirements (Local)
@ -41,14 +59,14 @@ its components.
The main codebase is a multi-module Scala project with multiple sub-modules. To start working with the project, from the
root directory run `sbt`. Most of the code can be found in the `modules` directory. The following modules are present:
* `root` - this is the parent project, if you run tasks here, it will run against all sub-modules
* [`core`](#core): core modules that are used by both the API and portal, such as cryptography implementations.
* [`api`](#api): the API is the main engine for all of VinylDNS. This is the most active area of the codebase, as
- `root` - this is the parent project, if you run tasks here, it will run against all sub-modules
- [`core`](#core): core modules that are used by both the API and portal, such as cryptography implementations.
- [`api`](#api): the API is the main engine for all of VinylDNS. This is the most active area of the codebase, as
everything else typically just funnels through the API.
* [`portal`](#portal): The portal is a user interface wrapper around the API. Most of the business rules, logic, and
- [`portal`](#portal): The portal is a user interface wrapper around the API. Most of the business rules, logic, and
processing can be found in the API. The
_only_ features in the portal not found in the API are creation of users and user authentication.
* [`docs`](#documentation): documentation for VinylDNS.
- [`docs`](#documentation): documentation for VinylDNS.
### Core
@ -56,76 +74,76 @@ Code that is used across multiple modules in the VinylDNS ecosystem live in `cor
#### Code Layout
* `src/main` - the main source code
* `src/test` - unit tests
- `src/main` - the main source code
- `src/test` - unit tests
### API
The API is the RESTful API for interacting with VinylDNS. The following technologies are used:
* [Akka HTTP](https://doc.akka.io/docs/akka-http/current/) - Used primarily for REST and HTTP calls.
* [FS2](https://functional-streams-for-scala.github.io/fs2/) - Used for backend change processing off of message queues.
- [Akka HTTP](https://doc.akka.io/docs/akka-http/current/) - Used primarily for REST and HTTP calls.
- [FS2](https://functional-streams-for-scala.github.io/fs2/) - Used for backend change processing off of message queues.
FS2 has back-pressure built in, and gives us tools like throttling and concurrency.
* [Cats Effect](https://typelevel.org/cats-effect/) - A replacement of `Future` with the `IO` monad
* [Cats](https://typelevel.org/cats) - Used for functional programming.
* [PureConfig](https://pureconfig.github.io/) - For loading configuration values.
- [Cats Effect](https://typelevel.org/cats-effect/) - A replacement of `Future` with the `IO` monad
- [Cats](https://typelevel.org/cats) - Used for functional programming.
- [PureConfig](https://pureconfig.github.io/) - For loading configuration values.
The API has the following dependencies:
* MySQL - the SQL database that houses the data
* SQS - for managing concurrent updates and enabling high-availability
* Bind9 - for testing integration with a real DNS system
- MySQL - the SQL database that houses the data
- SQS - for managing concurrent updates and enabling high-availability
- Bind9 - for testing integration with a real DNS system
#### Code Layout
The API code can be found in `modules/api`.
* `src/it` - integration tests
* `src/main` - the main source code
* `src/test` - unit tests
* `src/universal` - items that are packaged in the Docker image for the VinylDNS API
- `src/it` - integration tests
- `src/main` - the main source code
- `src/test` - unit tests
- `src/universal` - items that are packaged in the Docker image for the VinylDNS API
The package structure for the source code follows:
* `vinyldns.api.domain` - contains the core front-end logic. This includes things like the application services,
- `vinyldns.api.domain` - contains the core front-end logic. This includes things like the application services,
repository interfaces, domain model, validations, and business rules.
* `vinyldns.api.engine` - the back-end processing engine. This is where we process commands including record changes,
- `vinyldns.api.engine` - the back-end processing engine. This is where we process commands including record changes,
zone changes, and zone syncs.
* `vinyldns.api.protobuf` - marshalling and unmarshalling to and from protobuf to types in our system
* `vinyldns.api.repository` - repository implementations live here
* `vinyldns.api.route` - HTTP endpoints
- `vinyldns.api.protobuf` - marshalling and unmarshalling to and from protobuf to types in our system
- `vinyldns.api.repository` - repository implementations live here
- `vinyldns.api.route` - HTTP endpoints
### Portal
The project is built using:
* [Play Framework](https://www.playframework.com/documentation/2.6.x/Home)
* [AngularJS](https://angularjs.org/)
- [Play Framework](https://www.playframework.com/documentation/2.6.x/Home)
- [AngularJS](https://angularjs.org/)
The portal is _mostly_ a shim around the API. Most actions in the user interface are translated into API calls.
The features that the Portal provides that are not in the API include:
* Authentication against LDAP
* Creation of users - when a user logs in for the first time, VinylDNS automatically creates a user and new credentials
- Authentication against LDAP
- Creation of users - when a user logs in for the first time, VinylDNS automatically creates a user and new credentials
for them in the database with their LDAP information.
#### Code Layout
The portal code can be found in `modules/portal`.
* `app` - source code for portal back-end
* `models` - data structures that are used by the portal
* `views` - HTML templates for each web page
* `controllers` - logic for updating data
* `conf` - configurations and endpoint routes
* `public` - source code for portal front-end
* `css` - stylesheets
* `images` - images, including icons, used in the portal
* `js` - scripts
* `mocks` - mock JSON used in Grunt tests
* `templates` - modal templates
* `test` - unit tests for portal back-end
- `app` - source code for portal back-end
- `models` - data structures that are used by the portal
- `views` - HTML templates for each web page
- `controllers` - logic for updating data
- `conf` - configurations and endpoint routes
- `public` - source code for portal front-end
- `css` - stylesheets
- `images` - images, including icons, used in the portal
- `js` - scripts
- `mocks` - mock JSON used in Grunt tests
- `templates` - modal templates
- `test` - unit tests for portal back-end
### Documentation
@ -134,8 +152,8 @@ settings for the microsite are also configured in `build.sbt` of the project roo
#### Code Layout
* `src/main/resources` - Microsite resources and configurations
* `src/main/mdoc` - Content for microsite web pages
- `src/main/resources` - Microsite resources and configurations
- `src/main/mdoc` - Content for microsite web pages
## Running VinylDNS Locally
@ -146,20 +164,19 @@ README. However, VinylDNS can also be run in the foreground.
Before starting the API service, you can start the dependencies for local development:
```
cd test/api/integration
make build && make run-bg
```shell
quickstart/quickstart-vinyldns.sh --deps-only
```
This will start a container running in the background with necessary prerequisites.
Once the prerequisites are running, you can start up sbt by running `sbt` from the root directory.
* `project api` to change the sbt project to the API
* `reStart` to start up the API server
* Wait until you see the message `VINYLDNS SERVER STARTED SUCCESSFULLY` before working with the server
* To stop the VinylDNS server, run `reStop` from the api project
* To stop the dependent Docker containers: `utils/clean-vinyldns-containers.sh`
- `project api` to change the sbt project to the API
- `reStart` to start up the API server
- Wait until you see the message `VINYLDNS SERVER STARTED SUCCESSFULLY` before working with the server
- To stop the VinylDNS server, run `reStop` from the api project
- To stop the dependent Docker containers: `utils/clean-vinyldns-containers.sh`
See the [API Configuration Guide](https://www.vinyldns.io/operator/config-api) for information regarding API
configuration.
@ -167,9 +184,9 @@ configuration.
### Starting the Portal
To run the portal locally, you _first_ have to start up the VinylDNS API Server. This can be done by following the
instructions for [Staring the API Server](#Starting the API Server) or by using the QuickStart:
instructions for [Staring the API Server](#starting-the-api-server) or by using the QuickStart:
```
```shell
quickstart/quickstart-vinyldns.sh --api-only
```
@ -179,29 +196,6 @@ execute `;preparePortal; run`.
See the [Portal Configuration Guide](https://www.vinyldns.io/operator/config-portal) for information regarding portal
configuration.
### Loading test data
Normally the portal can be used for all VinylDNS requests. Test users are locked down to only have access to test zones,
which the portal connection modal has not been updated to incorporate. To connect to a zone with testuser, you will need
to use an alternative client and set `isTest=true` on the zone being connected to.
Use the vinyldns-js client (Note, you need Node installed):
```
git clone https://github.com/vinyldns/vinyldns-js.git
cd vinyldns-js
npm install
export VINYLDNS_API_SERVER=http://localhost:9000
export VINYLDNS_ACCESS_KEY_ID=testUserAccessKey
export VINYLDNS_SECRET_ACCESS_KEY=testUserSecretKey
npm run repl
> var groupId;
> vinyl.createGroup({"name": "test-group", "email":"test@test.com", members: [{id: "testuser"}], admins: [{id: "testuser"}]}).then(res => {groupId = res.id}).catch(err => {console.log(err)});
> vinyl.createZone ({name: "ok.", isTest: true, adminGroupId: groupId, email: "test@test.com"}).then(res => { console.log(res) }).catch(err => { console.log(err) })
You should now be able to see the zone in the portal at localhost:9001 when logged in as username=testuser password=testpassword
```
## Testing
### Unit Tests
@ -247,7 +241,7 @@ When adding new features, you will often need to write new functional tests that
To run functional tests you can simply execute the following commands:
```
```shell
build/func-test-api.sh
build/func-test-portal.sh
```
@ -259,9 +253,9 @@ These command will run the API functional tests and portal functional tests resp
To run functional tests you can simply execute `build/func-test-api.sh`, but if you'd like finer-grained control, you
can work with the `Makefile` in `test/api/functional`:
```
cd test/api/functional
make build && make run
```shell
# Build and then run the function test container
make -C test/api/functional build run
```
During iterative test development, you can use `make run-local` which will bind-mount the current functional tests in
@ -289,20 +283,20 @@ There aren't a lot, so it should be quick.
In the `modules/api/src/test/functional` directory are a few important files for you to be familiar with:
* `vinyl_python.py` - this provides the interface to the VinylDNS API. It handles signing the request for you, as well
- `vinyl_python.py` - this provides the interface to the VinylDNS API. It handles signing the request for you, as well
as building and executing the requests, and giving you back valid responses. For all new API endpoints, there should
be a corresponding function in the vinyl_client
* `utils.py` - provides general use functions that can be used anywhere in your tests. Feel free to contribute new
- `utils.py` - provides general use functions that can be used anywhere in your tests. Feel free to contribute new
functions here when you see repetition in the code
In the `modules/api/src/test/functional/tests` directory, we have directories / modules for different areas of the
application.
* `batch` - for managing batch updates
* `internal` - for internal endpoints (not intended for public consumption)
* `membership` - for managing groups and users
* `recordsets` - for managing record sets
* `zones` - for managing zones
- `batch` - for managing batch updates
- `internal` - for internal endpoints (not intended for public consumption)
- `membership` - for managing groups and users
- `recordsets` - for managing record sets
- `zones` - for managing zones
##### Functional Test Context

View File

@ -27,12 +27,6 @@ The offline root key and repository keys are managed by the core maintainer team
* api key: used to sign tagged images in vinyldns/api
* portal key: used to sign tagged images in vinyldns/portal
These keys are named in a <hash>.key format, e.g. 5526ecd15bd413e08718e66c440d17a28968d5cd2922b59a17510da802ca6572.key,
do not change the names of the keys.
Docker expects these keys to be saved in `~/.docker/trust/private`. Each key is encrypted with a passphrase, that you
must have available when pushing an image.
## Release Process
The release process is automated by GitHub Actions.

View File

@ -1,10 +1,9 @@
![Build](https://github.com/vinyldns/vinyldns/workflows/Continuous%20Integration/badge.svg)
[![CodeCov ](https://codecov.io/gh/vinyldns/vinyldns/branch/master/graph/badge.svg)](https://codecov.io/gh/vinyldns/vinyldns)
[![License](https://img.shields.io/github/license/vinyldns/vinyldns)](https://github.com/vinyldns/vinyldns/blob/master/LICENSE)
[![conduct](https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg)](https://github.com/vinyldns/vinyldns/blob/master/CODE_OF_CONDUCT.md)
<p align="left">
<a href="http://www.vinyldns.io/">
<a href="https://www.vinyldns.io/">
<img
alt="VinylDNS"
src="img/vinyldns_optimized.svg"
@ -38,6 +37,9 @@ Integration is simple with first-class language support including:
## Table of Contents
- [Quickstart](#quickstart)
- [Things to Try in the Portal](#things-to-try-in-the-portal)
- [Verifying Your Changes](#verifying-your-changes)
- [Other things to note](#other-things-to-note)
- [Code of Conduct](#code-of-conduct)
- [Developer Guide](#developer-guide)
- [Contributing](#contributing)
@ -56,29 +58,32 @@ VinylDNS on your machine with docker:
1. Navigate to repo: `cd vinyldns`
1. Run `./quickstart/quickstart-vinyldns.sh`. This will start up the api at `localhost:9000` and the portal
at `localhost:9001`
1. See [Developer Guide](DEVELOPER_GUIDE.md#loading-test-data) for how to load a test DNS zone
1. See [Things to Try in the Portal](#things-to-try-in-the-portal) for getting familiar with the Portal
1. To stop the local setup, run `./utils/clean-vinyldns-containers.sh`.
There exist several clients at <https://github.com/vinyldns> that can be used to make API requests, using the
endpoint `http://localhost:9000`
## Things to try in the portal
## Things to Try in the Portal
1. View the portal at <http://localhost:9001> in a web browser
2. Login with the credentials `professor` and `professor`
3. Navigate to the `groups` tab: <http://localhost:9001/groups>
4. Click on the **New Group** button and create a new group, the group id is the uuid in the url after you view the
group
5. View zones you connected to in the `zones` tab: <http://localhost:9001/zones>. For a quick test, create a new zone
named `ok` with an email of `test@test.com` and choose a group you created from the previous step. (Note,
see [Developer Guide](DEVELOPER_GUIDE.md#loading-test-data) for creating a zone)
6. You will see that some records are preloaded in the zoned already, this is because these records are preloaded in the
5. Connect a zone by going to the `zones` tab: <http://localhost:9001/zones>.
1. Click the `-> Connect` button
2. For `Zone Name` enter `ok` with an email of `test@test.com`
3. For `Admin Group`, choose a group you created from the previous step
4. Leave everything else as-is and click the `Connect` button at the bottom of the form
6. A new zone `ok` should appear in your `My Zones` tab _(you may need to refresh your browser)_
7. You will see that some records are preloaded in the zone already, this is because these records are preloaded in the
local docker DNS server and VinylDNS automatically syncs records with the backend DNS server upon zone connection
7. From here, you can create DNS record sets in the **Manage Records** tab, and manage zone settings and ***ACL rules***
8. From here, you can create DNS record sets in the **Manage Records** tab, and manage zone settings and ***ACL rules***
in the **Manage Zone** tab
8. To try creating a DNS record, click on the **Create Record Set** button under
9. To try creating a DNS record, click on the **Create Record Set** button under
Records, `Record Type = A, Record Name = my-test-a, TTL = 300, IP Addressess = 1.1.1.1`
9. Click on the **Refresh** button under Records, you should see your new record created
10. Click on the **Refresh** button under Records, you should see your new record created
### Verifying Your Changes
@ -96,7 +101,7 @@ This tells `dig` to use `127.0.0.1` as the resolver on port `19001`. The `+short
verbose. Finally, the record we're looking up is `my-test-a.ok`. You can see the returned output of `1.1.1.1` matches
the record data we entered.
## Other things to note
### Other things to note
1. Upon connecting to a zone for the first time, a zone sync is executed to provide VinylDNS a copy of the records in
the zone

View File

@ -1,56 +1,57 @@
# System Design
## Table of Contents
- [Components](#components)
- [Process Flow](#process-flow)
- [Integration](#integration)
## Components
The following diagram illustrates the major components in the VinylDNS ecosystem and the external systems they interact with.
The following diagram illustrates the major components in the VinylDNS ecosystem and the external systems they interact
with.
![VinylDNS Architecture Diagram](img/VinylDNS_overview.png)
![VinylDNS Architecture Diagram](img/vinyldns_overview.png)
* API - RESTful endpoints to allow interaction with VinylDNS
* Database - stores information that the VinylDNS application needs
* DNS servers - communicates DNS changes and resolves DNS records
* Message queue - temporarily stores DNS requests for processing
* LDAP Service - application protocol used to authenticate user access to the VinylDNS portal
* Portal - graphical user interface to interact with the VinylDNS API
* Tooling - external libraries and utilities used to interact with the VinylDNS API
| Component | Description |
|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Portal | Web user interface to interact with the VinylDNS API |
| API | RESTful endpoints to allow interaction with VinylDNS |
| API Worker Nodes | These are API components with `processing-disabled` set to `false` (see [documentation](https://www.vinyldns.io/operator/config-api.html#processing-disabled)) |
| Message queue | Queue for DNS commands to enable flow control to the DNS backends (see [documentation](https://www.vinyldns.io/operator/pre.html#message-queues)) |
| Database | Stores information about users, membership, and DNS records |
| DNS Backend(s) | The DNS backend servers which VinylDNS will query and update. |
| LDAP Service | The optional LDAP service that VinylDNS can be configured to communicate with (see [documentation](https://www.vinyldns.io/operator/setup-ldap.html#setup-ldap)) |
## Process Flow
1. LDAP service authenticates user credentials and grants access to the portal.
1. If the user is accessing the portal for the first time, VinylDNS credentials are generated and stored.
1. User navigates portal or uses integration tooling to generate a signed API request.
1. When the API receives a request, it loads the credentials for the calling user from the database and validates the request signature to ensure that the request was not modified in transit.
1. When the API receives a request, it loads the credentials for the calling user from the database and validates the
request signature to ensure that the request was not modified in transit.
1. The request is then validated to ensure that:
- the request data is correct
- the request passes all validation checks
- the user has access to make the change
1. Assuming the request is in good order, the request is put on a message queue for handling.
1. One of the VinylDNS API server instances pulls the message from the queue for processing. For record changes, a DDNS message is issued to the DNS backend server.
1. When the message completes processing, it is removed from the message queue. The changes are applied to the VinylDNS database along with an audit record for the request.
1. One of the VinylDNS API server instances pulls the message from the queue for processing. For record changes, a DDNS
message is issued to the DNS backend server.
1. When the message completes processing, it is removed from the message queue. The changes are applied to the VinylDNS
database along with an audit record for the request.
## Integration
Integrating with VinylDNS is simple since each API endpoint is effectively a distinct DNS operation (eg. create record, update record, delete record, etc.). The only requirement for sending a request is generating the correct AWS SIG4 signature without content length and providing the corresponding HTTP headers so that VinylDNS can verify it. See [API Authentication](https://www.vinyldns.io/api/auth-mechanism.html) for more details.
Integrating with VinylDNS is simple since each API endpoint is effectively a distinct DNS operation (eg. create record,
update record, delete record, etc.). The only requirement for sending a request is generating the correct AWS SIG4
signature without content length and providing the corresponding HTTP headers so that VinylDNS can verify it.
See [API Authentication](https://www.vinyldns.io/api/auth-mechanism.html) for more details.
The current tooling available to perform VinylDNS API requests include:
* [go-vinyldns](https://github.com/vinyldns/go-vinyldns) - Golang client package
* [vinyldns-cli](https://github.com/vinyldns/vinyldns-cli) - command line utility written in Golang
* [terraform-provider-vinyldns](https://github.com/vinyldns/terraform-provider-vinyldns) - A [Terraform](https://terraform.io/) provider for VinylDNS
* [vinyldns-cli](https://github.com/vinyldns/vinyldns-cli) - Command line utility written in Golang
* [vinyldns-java](https://github.com/vinyldns/vinyldns-java) - Java client
* [vinyldns-js](https://github.com/vinyldns/vinyldns-js) - JavaScript client
* [vinyldns-python](https://github.com/vinyldns/vinyldns-python) - Python client library
* [vinyldns-ruby](https://github.com/vinyldns/vinyldns-ruby) - Ruby gem

View File

@ -5,6 +5,7 @@ import org.scalafmt.sbt.ScalafmtPlugin._
import scoverage.ScoverageKeys.{coverageFailOnMinimum, coverageMinimum}
import scala.language.postfixOps
import scala.sys.env
import scala.util.Try
lazy val IntegrationTest = config("it").extend(Test)
@ -16,9 +17,9 @@ lazy val sharedSettings = Seq(
organizationName := "Comcast Cable Communications Management, LLC",
startYear := Some(2018),
licenses += ("Apache-2.0", new URL("https://www.apache.org/licenses/LICENSE-2.0.txt")),
maintainer := "VinylDNS Maintainers <vinyldns-core@googlegroups.com>",
maintainer := "VinylDNS Maintainers <vinyldns-core@googlegroups.com>",
scalacOptions ++= scalacOptionsByV(scalaVersion.value),
scalacOptions in (Compile, doc) += "-no-link-warnings",
scalacOptions in(Compile, doc) += "-no-link-warnings",
// Use wart remover to eliminate code badness
wartremoverErrors := (
if (getPropertyFlagOrDefault("build.lintOnCompile", true))
@ -31,13 +32,13 @@ lazy val sharedSettings = Seq(
Wart.ExplicitImplicitTypes
)
else Seq.empty
),
),
// scala format
scalafmtOnCompile := getPropertyFlagOrDefault("build.scalafmtOnCompile", false),
// coverage options
coverageMinimum := 85,
coverageFailOnMinimum := true,
coverageHighlighting := true
coverageHighlighting := true,
)
lazy val testSettings = Seq(
@ -287,7 +288,6 @@ lazy val docs = (project in file("modules/docs"))
.settings(docSettings)
def getPropertyFlagOrDefault(name: String, value: Boolean): Boolean =
sys.props.get(name).flatMap(propValue => Try(propValue.toBoolean).toOption).getOrElse(value)

View File

@ -7,7 +7,7 @@ TEST_LOGIN=false
# API Settings
REST_PORT=9000
SQS_ENDPOINT=http://vinyldns-integration:19003
SQS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
MYSQL_ENDPOINT=vinyldns-integration:19002
DEFAULT_DNS_ADDRESS=vinyldns-integration:19001

View File

@ -49,7 +49,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1"
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
@ -58,7 +58,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1"
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
@ -71,7 +71,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1"
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
@ -80,7 +80,7 @@ vinyldns {
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1"
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"

View File

@ -4,4 +4,4 @@ set -euo pipefail
DIR=$(cd -P -- "$(dirname -- "$0")" && pwd -P)
cd "$DIR/../test/api/integration"
make build DOCKER_PARAMS="--build-arg SKIP_API_BUILD=true" && make run-local WITH_ARGS="sbt" DOCKER_PARAMS="-e RUN_SERVICES=none"
make build DOCKER_PARAMS="--build-arg SKIP_API_BUILD=true" && make run-local WITH_ARGS="sbt" DOCKER_PARAMS="-e RUN_SERVICES=none --env-file \"$DIR/../test/api/integration/.env.integration\""

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

BIN
img/vinyldns_overview.png Executable file

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

View File

@ -1,63 +1,268 @@
akka {
loglevel = "OFF"
log-dead-letters-during-shutdown = off
log-dead-letters = 0
logger-startup-timeout = 60s
}
vinyldns {
base-version = "0.0.0-local-dev"
version = ${vinyldns.base-version} # default to the base version if not overridden
version = ${?VINYLDNS_VERSION} # override the base version via env var
# How often to any particular zone can be synchronized in milliseconds
sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
}
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
rest {
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
approved-name-servers = [
"172.17.42.1."
"172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql {
settings {
# see https://github.com/brettwooldridge/HikariCP
connection-timeout-millis = 1000
idle-timeout = 10000
max-lifetime = 600000
maximum-pool-size = 5
minimum-idle = 1
my-sql-properties = {
cachePrepStmts=true
prepStmtCacheSize=250
prepStmtCacheSqlLimit=2048
rewriteBatchedStatements=true
}
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
# no additional settings for now
}
batch-change {
# no additional settings for now
}
user {
}
record-set {
}
group {
}
membership {
}
group-change {
}
zone-change {
}
record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
@ -66,31 +271,65 @@ vinyldns {
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252",
"192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
"fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
]
}
# types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
global-acl-rules = [
{
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared[0-9]{1}."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
}
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
}
akka {
loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s
email.settings.smtp {
port = 19025
actor {
provider = "akka.actor.LocalActorRefProvider"
}
}
# Global settings
scalikejdbc.global.loggingSQLAndTime.enabled=true
scalikejdbc.global.loggingSQLAndTime.logLevel=error
scalikejdbc.global.loggingSQLAndTime.warningEnabled=true
scalikejdbc.global.loggingSQLAndTime.warningThresholdMillis=1000
scalikejdbc.global.loggingSQLAndTime.warningLogLevel=warn
scalikejdbc.global.loggingSQLAndTime.singleLineMode=false
scalikejdbc.global.loggingSQLAndTime.printUnprocessedStackTrace=false
scalikejdbc.global.loggingSQLAndTime.stackTraceDepth=10
akka.http {
server {
# The time period within which the TCP binding process must be completed.
# Set to `infinite` to disable.
bind-timeout = 5s
# Show verbose error messages back to the client
verbose-error-messages = on
}
parsing {
# Spray doesn't like the AWS4 headers
illegal-header-warnings = on
}
}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

View File

@ -34,15 +34,16 @@ import vinyldns.core.domain.zone.{Algorithm, Zone, ZoneConnection}
class DnsBackendIntegrationSpec extends AnyWordSpec with Matchers {
private val testConnection = ZoneConnection(
"test",
"test",
"vinyldns.",
"vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001",
sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001"),
Algorithm.HMAC_MD5
)
"DNSBackend" should {
"connect to a zone without a tsig key for transfer or update" in {
val config =
DnsBackendConfig(
"test",

View File

@ -51,7 +51,7 @@ class ZoneViewLoaderIntegrationSpec extends AnyWordSpec with Matchers {
"vinyldns.",
"vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001"
sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001")
)
),
transferConnection =
@ -84,7 +84,7 @@ class ZoneViewLoaderIntegrationSpec extends AnyWordSpec with Matchers {
"vinyldns.",
"vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001"
sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001")
)
),
transferConnection = Some(
@ -92,7 +92,7 @@ class ZoneViewLoaderIntegrationSpec extends AnyWordSpec with Matchers {
"vinyldns.",
"vinyldns.",
"nzisn+4G2ldMn0q1CV3vsg==",
"127.0.0.1:19001"
sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001")
)
)
)

View File

@ -16,7 +16,7 @@
package vinyldns.api.notifier.sns
import cats.effect.IO
import cats.effect.{IO, Timer}
import com.amazonaws.auth.{AWSStaticCredentialsProvider, BasicAWSCredentials}
import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration
import com.amazonaws.services.sns.AmazonSNSClientBuilder
@ -26,6 +26,7 @@ import org.joda.time.DateTime
import org.json4s.DefaultFormats
import org.json4s.jackson.JsonMethods._
import org.scalatest.matchers.should.Matchers
import org.scalatest.time.SpanSugar.convertIntToGrainOfTime
import org.scalatest.wordspec.AnyWordSpecLike
import vinyldns.api.MySqlApiIntegrationSpec
import vinyldns.core.TestMembershipData._
@ -34,6 +35,8 @@ import vinyldns.core.domain.record.{AData, RecordType}
import vinyldns.core.notifier._
import vinyldns.mysql.MySqlIntegrationSpec
import scala.concurrent.ExecutionContext
class SnsNotifierIntegrationSpec
extends MySqlApiIntegrationSpec
with MySqlIntegrationSpec
@ -43,7 +46,7 @@ class SnsNotifierIntegrationSpec
import vinyldns.api.domain.DomainValidations._
implicit val formats = DefaultFormats
implicit val timer: Timer[IO] = IO.timer(ExecutionContext.global)
val snsConfig: Config = ConfigFactory.load().getConfig("vinyldns.sns.settings")
"Sns Notifier" should {
@ -91,7 +94,7 @@ class SnsNotifierIntegrationSpec
val sqs = AmazonSQSClientBuilder
.standard()
.withEndpointConfiguration(
new EndpointConfiguration("http://127.0.0.1:19003", "us-east-1")
new EndpointConfiguration(sys.env.getOrElse("SNS_SERVICE_ENDPOINT","http://vinyldns-integration:19003"), "us-east-1")
)
.withCredentials(credentialsProvider)
.build()
@ -103,7 +106,7 @@ class SnsNotifierIntegrationSpec
notifier <- new SnsNotifierProvider()
.load(NotifierConfig("", snsConfig), userRepository)
_ <- notifier.notify(Notification(batchChange))
_ <- IO { Thread.sleep(100) }
_ <- IO.sleep(1.seconds)
messages <- IO { sqs.receiveMessage(queueUrl).getMessages }
_ <- IO {
sns.deleteTopic(topic)

View File

@ -39,7 +39,7 @@ import scala.collection.JavaConverters._
import scala.util.matching.Regex
class Route53ApiIntegrationSpec
extends AnyWordSpec
extends AnyWordSpec
with ScalaFutures
with Matchers
with MockitoSugar
@ -49,6 +49,7 @@ class Route53ApiIntegrationSpec
with MySqlApiIntegrationSpec {
private val testZone = Zone("example.com.", "test@test.com", backendId = Some("test"))
private def testConnection: Route53Backend =
Route53Backend
.load(
@ -56,7 +57,7 @@ class Route53ApiIntegrationSpec
"test",
Some("access"),
Some("secret"),
"http://127.0.0.1:19003",
sys.env.getOrElse("R53_SERVICE_ENDPOINT", "http://localhost:19003"),
"us-east-1"
)
)

View File

@ -1,15 +1,317 @@
################################################################################################################
# This configuration is used primarily when running re-start or starting Vinyll locally. The configuration
# presumes a stand-alone Vinyll server with no backend services.
################################################################################################################
akka {
loglevel = "ERROR"
vinyldns {
base-version = "0.0.0-local-dev"
version = ${vinyldns.base-version} # default to the base version if not overridden
version = ${?VINYLDNS_VERSION} # override the base version via env var
# The following settings are required to have Akka logging output to SLF4J and logback; without
# these, akka will output to STDOUT
# How often to any particular zone can be synchronized in milliseconds
sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
}
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
rest {
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
approved-name-servers = [
"172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql {
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
}
batch-change {
}
user {
}
record-set {
}
zone-change {
}
record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252",
"192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
]
}
global-acl-rules = [
{
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared[0-9]{1}."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
}
akka {
loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
}
}
akka.http {
@ -28,143 +330,6 @@ akka.http {
}
}
vinyldns {
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
max-retries = 100 # Max retries for message on queue; currently only applies to MySqlMessageQueue
settings {
# AWS access key and secret.
access-key = "x"
secret-key = "x"
}
}
data-stores = ["mysql"]
mysql {
settings {
# see https://github.com/brettwooldridge/HikariCP
connection-timeout-millis = 1000
idle-timeout = 10000
max-lifetime = 600000
maximum-pool-size = 5
minimum-idle = 1
register-mbeans = true
}
repositories {
zone {
# no additional settings for now
}
batch-change {
# no additional settings for now
}
user {
}
record-set {
}
group {
}
membership {
}
group-change {
}
zone-change {
}
record-change {
}
}
}
sync-delay = 10000 # 10 second delay for resyncing zone
batch-change-limit = 1000 # Max change limit per batch request
# this key is used in order to encrypt/decrypt DNS TSIG keys. We use this dummy one for test purposes, this
# should be overridden with a real value that is hidden for production deployment
crypto {
type = "vinyldns.core.crypto.JavaCrypto"
secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.2.254",
"192.0.2.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
]
}
# types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
backends = [
{
id = "func-test-backend"
zone-connection {
name = "vinyldns."
key-name = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "127.0.0.1:19001"
}
transfer-connection {
name = "vinyldns."
key-name = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "127.0.0.1:19001"
}
}
]
multi-record-batch-change-enabled = true
# feature flag for manual batch review
manual-batch-review-enabled = true
scheduled-changes-enabled = true
global-acl-rules = [
{
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok."]
}
]
}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

View File

@ -148,16 +148,24 @@ vinyldns {
defaultZoneConnection {
name = "vinyldns."
name= ${?DEFAULT_DNS_KEY_NAME}
keyName = "vinyldns."
keyName= ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primaryServer = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
defaultTransferConnection {
name = "vinyldns."
name= ${?DEFAULT_DNS_KEY_NAME}
keyName = "vinyldns."
keyName= ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primaryServer = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
batch-change-limit = 1000

View File

@ -1,6 +1,6 @@
.__ .__ .___
___ _|__| ____ ___.__.| | __| _/____ ______
\ \/ / |/ < | || | / __ |/ \ / ___/
\ /| | | \___ || |__/ /_/ | | \\___ \
\_/ |__|___| / ____||____/\____ |___| /____ >
\/\/ \/ \/ \/
_ ___ ______ _ _______
| | / (_)___ __ __/ / __ \/ | / / ___/
| | / / / __ \/ / / / / / / / |/ /\__ \
| |/ / / / / / /_/ / / /_/ / /| /___/ /
|___/_/_/ /_/\__, /_/_____/_/ |_//____/
/____/

View File

@ -1,30 +1,227 @@
akka {
loglevel = "OFF"
loggers = ["akka.testkit.TestEventListener"]
log-dead-letters-during-shutdown = off
log-dead-letters = 0
test.timefactor = 5
logger-startup-timeout = 30s
}
vinyldns {
active-node-count = 3
sync-delay = 10000 # 10 second delay for resyncing zone
color = "blue"
base-version = "0.0.0-local-dev"
version = ${vinyldns.base-version} # default to the base version if not overridden
version = ${?VINYLDNS_VERSION} # override the base version via env var
# How often to any particular zone can be synchronized in milliseconds
sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
}
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
notifiers = ["test-notifier"]
test-notifier {
class-name = "someclass"
settings {
value = "test"
}
}
rest {
host = "127.0.0.1"
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
# this key is used in order to encrypt/decrypt DNS TSIG keys. We use this dummy one for test purposes, this
# should be overridden with a real value that is hidden for production deployment
crypto {
type = "vinyldns.core.crypto.JavaCrypto"
secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
}
approved-name-servers = [
"some.test.ns."
"172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
mysql {
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
}
batch-change {
}
user {
}
record-set {
}
zone-change {
}
record-change {
}
group {
}
group-change {
}
membership {
}
}
}
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
@ -33,6 +230,7 @@ vinyldns {
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
@ -48,80 +246,99 @@ vinyldns {
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2"
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.needs.review."
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
}
# types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# used for testing only
string-list-test = ["test"]
mysql.repositories {
zone {
# no additional settings for now
}
batch-change {
# no additional settings for now
}
user {
}
record-set {
}
group {
}
membership {
}
group-change {
}
zone-change {
}
record-change {
}
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252",
"192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
]
}
notifiers = ["test-notifier"]
test-notifier {
class-name = "someclass"
settings {
value = "test"
}
}
backends = [
global-acl-rules = [
{
id = "test"
zone-connection {
name = "zoneconn."
key-name = "vinyldns."
key = "test-key"
primary-server = "127.0.0.1:19001"
}
transfer-connection {
name = "transferconn."
key-name = "vinyldns."
key = "test-key"
primary-server = "127.0.0.1:19001"
}
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared[0-9]{1}."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
manual-batch-review-enabled = true
scheduled-changes-enabled = true
}
akka {
loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
}
}
akka.http {
server {
# The time period within which the TCP binding process must be completed.
# Set to `infinite` to disable.
bind-timeout = 5s
# Show verbose error messages back to the client
verbose-error-messages = on
}
parsing {
# Spray doesn't like the AWS4 headers
illegal-header-warnings = on
}
}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

View File

@ -16,15 +16,18 @@
package vinyldns.api.config
import cats.effect.{ContextShift, IO}
import org.scalatest.BeforeAndAfterAll
import org.scalatest.matchers.should.Matchers
import org.scalatest.wordspec.AnyWordSpec
import vinyldns.api.backend.dns.DnsBackendProviderConfig
import vinyldns.core.domain.zone.ZoneConnection
import vinyldns.core.repository.RepositoryName._
class VinylDNSConfigSpec extends AnyWordSpec with Matchers with BeforeAndAfterAll {
private val underTest: VinylDNSConfig = VinylDNSConfig.load().unsafeRunSync()
private implicit val cs: ContextShift[IO] = IO.contextShift(scala.concurrent.ExecutionContext.global)
"VinylDNSConfig" should {
"load the rest config" in {
@ -63,27 +66,19 @@ class VinylDNSConfigSpec extends AnyWordSpec with Matchers with BeforeAndAfterAl
notifierConfigs.head.settings.getString("value").shouldBe("test")
}
"load default keys" in {
val defaultConn =
ZoneConnection("vinyldns.", "vinyldns.", "nzisn+4G2ldMn0q1CV3vsg==", "127.0.0.1:19001")
underTest.configuredDnsConnections.defaultZoneConnection
.decrypted(underTest.crypto) shouldBe
defaultConn
underTest.configuredDnsConnections.defaultTransferConnection
.decrypted(underTest.crypto) shouldBe
defaultConn
}
"load specified backends" in {
val zc = ZoneConnection("zoneconn.", "vinyldns.", "test-key", "127.0.0.1:19001")
val tc = zc.copy(name = "transferconn.")
val zc = ZoneConnection("vinyldns.", "vinyldns.", "nzisn+4G2ldMn0q1CV3vsg==", sys.env.getOrElse("DEFAULT_DNS_ADDRESS", "127.0.0.1:19001"))
val tc = zc.copy()
val backends = underTest.configuredDnsConnections.dnsBackends
val backends = underTest.backendConfigs.backendProviders
backends.length shouldBe 1
backends.head.id shouldBe "test"
backends.head.zoneConnection.decrypted(underTest.crypto) shouldBe zc
backends.head.transferConnection.decrypted(underTest.crypto) shouldBe tc
val config = DnsBackendProviderConfig.load(backends.head.settings).unsafeRunSync()
config.backends.length shouldBe 2
config.backends.head.id shouldBe "default"
config.backends.head.zoneConnection.decrypted(underTest.crypto) shouldBe zc
config.backends.head.transferConnection.get.decrypted(underTest.crypto) shouldBe tc
}
}
}

View File

@ -1,124 +1,317 @@
# The default application.conf is not intended to be used in production. It assumes a docker-compose
# setup for all of the services. Provide your own application.conf on the docker mount with your
# own settings
vinyldns {
# Should be provided in the start up script via an environment
base-version = "unset"
base-version = "0.0.0-local-dev"
version = ${vinyldns.base-version} # default to the base version if not overridden
version = ${?VINYLDNS_VERSION} # override the base version via env var
queue.settings.service-endpoint = "http://vinyldns-elasticmq:9324/"
# How often to any particular zone can be synchronized in milliseconds
sync-delay = 10000
sync-delay = ${?SYNC_DELAY}
# If we should start up polling for change requests, set this to false for the inactive cluster
processing-disabled = false
processing-disabled = ${?PROCESSING_DISABLED}
# Number of records that can be in a zone
max-zone-size = 60000
max-zone-size = ${?MAX_ZONE_SIZE}
# Types of unowned records that users can access in shared zones
shared-approved-types = ["A", "AAAA", "CNAME", "PTR", "TXT"]
# Batch change settings
batch-change-limit = 1000
batch-change-limit = ${?BATCH_CHANGE_LIMIT}
manual-batch-review-enabled = true
manual-batch-review-enabled = ${?MANUAL_BATCH_REVIEW_ENABLED}
scheduled-changes-enabled = true
scheduled-changes-enabled = ${?SCHEDULED_CHANGES_ENABLED}
multi-record-batch-change-enabled = true
multi-record-batch-change-enabled = ${?MULTI_RECORD_BATCH_CHANGE_ENABLED}
# configured backend providers
backend {
# Use "default" when dns backend legacy = true
# otherwise, use the id of one of the connections in any of your backends
default-backend-id = "default"
# this is where we can save additional backends
backend-providers = [
{
class-name = "vinyldns.api.backend.dns.DnsBackendProviderLoader"
settings = {
legacy = false
backends = [
{
id = "default"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
},
{
id = "func-test-backend"
zone-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
}
transfer-connection = {
name = "vinyldns."
key-name = "vinyldns."
key-name = ${?DEFAULT_DNS_KEY_NAME}
key = "nzisn+4G2ldMn0q1CV3vsg=="
key = ${?DEFAULT_DNS_KEY_SECRET}
primary-server = "127.0.0.1:19001"
primary-server = ${?DEFAULT_DNS_ADDRESS}
},
tsig-usage = "always"
}
]
}
}
]
}
queue {
class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
messages-per-poll = 10
polling-interval = 250.millis
settings {
# AWS access key and secret.
access-key = "test"
access-key = ${?AWS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?AWS_SECRET_ACCESS_KEY}
# Regional endpoint to make your requests (eg. 'us-west-2', 'us-east-1', etc.). This is the region where your queue is housed.
signing-region = "us-east-1"
signing-region = ${?SQS_REGION}
# Endpoint to access queue
service-endpoint = "http://vinyldns-integration:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"
queue-name = ${?SQS_QUEUE_NAME}
}
}
email {
class-name = "vinyldns.api.notifier.email.EmailNotifierProvider"
class-name = ${?EMAIL_CLASS_NAME}
settings = {
from = "VinylDNS <do-not-reply@vinyldns.io>"
from = ${?EMAIL_FROM}
}
}
sns {
class-name = "vinyldns.apadi.notifier.sns.SnsNotifierProvider"
class-name = ${?SNS_CLASS_NAME}
settings {
topic-arn = "arn:aws:sns:us-east-1:000000000000:batchChanges"
topic-arn = ${?SNS_TOPIC_ARN}
access-key = "test"
access-key = ${?SNS_ACCESS_KEY}
secret-key = "test"
secret-key = ${?SNS_SECRET_KEY}
service-endpoint = "http://vinyldns-integration:19003"
service-endpoint = ${?SNS_SERVICE_ENDPOINT}
signing-region = "us-east-1"
signing-region = ${?SNS_REGION}
}
}
# host and port the server binds to. This should not be changed
rest {
host = "0.0.0.0"
port = 9000
port=${?API_SERVICE_PORT}
}
# the delay between zone syncs so we are not syncing too often
sync-delay = 10000
# crypto settings for symmetric cryptography of secrets in the system
# Note: for production systems secrets should not live in plain text in a file
approved-name-servers = [
"172.17.42.1.",
"ns1.parent.com."
"ns1.parent.com1."
"ns1.parent.com2."
"ns1.parent.com3."
"ns1.parent.com4."
]
# Note: This MUST match the Portal or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
data-stores = ["mysql"]
# default settings point to the setup from docker compose
mysql {
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
# these must be overridden to use MYSQL for production use
# assumes a docker or mysql instance running locally
name = "vinyldns"
driver = "org.mariadb.jdbc.Driver"
migration-url = "jdbc:mariadb://vinyldns-mysql:3306/?user=root&password=pass"
url = "jdbc:mariadb://vinyldns-mysql:3306/vinyldns?user=root&password=pass"
user = "root"
password = "pass"
# see https://github.com/brettwooldridge/HikariCP
connection-timeout-millis = 1000
max-lifetime = 600000
maximum-pool-size = 20
register-mbeans = true
name = ${?DATABASE_NAME}
driver = "org.h2.Driver"
driver = ${?JDBC_DRIVER}
migration-url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
migration-url = ${?JDBC_MIGRATION_URL}
url = "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
url = ${?JDBC_URL}
user = "sa"
user = ${?JDBC_USER}
password = ""
password = ${?JDBC_PASSWORD}
}
# TODO: Remove the need for these useless configuration blocks
repositories {
zone {
# no additional settings for now
}
batch-change {
# no additional settings for now
}
user {
# no additional settings for now
}
record-set {
# no additional settings for now
}
group {
# no additional settings for now
}
membership {
# no additional settings for now
}
group-change {
# no additional settings for now
}
zone-change {
# no additional settings for now
}
record-change {
# no additional settings for now
}
group {
}
group-change {
}
membership {
}
}
}
# the DDNS connection information for the default dns backend
defaultZoneConnection {
name = "vinyldns."
keyName = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primaryServer = "localhost:19001"
backends = []
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.2.252",
"192.0.2.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0"
]
}
# the AXFR connection information for the default dns backend
defaultTransferConnection {
name = "vinyldns."
keyName = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primaryServer = "localhost:19001"
# FQDNs / IPs / zone names that require manual review upon submission in batch change interface
# domain-list used for all record types except PTR
# ip-list used exclusively for PTR records
manual-review-domains = {
domain-list = [
"needs-review.*"
]
ip-list = [
"192.0.1.254",
"192.0.1.255",
"192.0.2.254",
"192.0.2.255",
"192.0.3.254",
"192.0.3.255",
"192.0.4.254",
"192.0.4.255",
"fd69:27cc:fe91:0:0:0:ffff:1",
"fd69:27cc:fe91:0:0:0:ffff:2",
"fd69:27cc:fe92:0:0:0:ffff:1",
"fd69:27cc:fe92:0:0:0:ffff:2",
"fd69:27cc:fe93:0:0:0:ffff:1",
"fd69:27cc:fe93:0:0:0:ffff:2",
"fd69:27cc:fe94:0:0:0:ffff:1",
"fd69:27cc:fe94:0:0:0:ffff:2"
]
zone-name-list = [
"zone.requires.review."
"zone.requires.review1."
"zone.requires.review2."
"zone.requires.review3."
"zone.requires.review4."
]
}
backends = [
# FQDNs / IPs that cannot be modified via VinylDNS
# regex-list used for all record types except PTR
# ip-list used exclusively for PTR records
high-value-domains = {
regex-list = [
"high-value-domain.*" # for testing
]
ip-list = [
# using reverse zones in the vinyldns/bind9 docker image for testing
"192.0.1.252",
"192.0.1.253",
"192.0.2.252",
"192.0.2.253",
"192.0.3.252",
"192.0.3.253",
"192.0.4.252",
"192.0.4.253",
"fd69:27cc:fe91:0:0:0:0:ffff",
"fd69:27cc:fe91:0:0:0:ffff:0",
"fd69:27cc:fe92:0:0:0:0:ffff",
"fd69:27cc:fe92:0:0:0:ffff:0",
"fd69:27cc:fe93:0:0:0:0:ffff",
"fd69:27cc:fe93:0:0:0:ffff:0",
"fd69:27cc:fe94:0:0:0:0:ffff",
"fd69:27cc:fe94:0:0:0:ffff:0"
]
}
global-acl-rules = [
{
id = "func-test-backend"
zone-connection {
name = "vinyldns."
key-name = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "localhost:19001"
}
transfer-connection {
name = "vinyldns."
key-name = "vinyldns."
key = "nzisn+4G2ldMn0q1CV3vsg=="
primary-server = "localhost:19001"
}
group-ids: ["global-acl-group-id"],
fqdn-regex-list: [".*shared[0-9]{1}."]
},
{
group-ids: ["another-global-acl-group"],
fqdn-regex-list: [".*ok[0-9]{1}."]
}
]
# the max number of changes in a single batch change. Change carefully as this has performance
# implications
batch-change-limit = 1000
}
# Akka settings, these should not need to be modified unless you know akka http really well.
akka {
loglevel = "INFO"
loggers = ["akka.event.slf4j.Slf4jLogger"]
logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
logger-startup-timeout = 30s
actor {
provider = "akka.actor.LocalActorRefProvider"
}
}
akka.http {
@ -132,7 +325,11 @@ akka.http {
}
parsing {
# akka-http doesn't like the AWS4 headers
# Spray doesn't like the AWS4 headers
illegal-header-warnings = on
}
}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

View File

@ -52,7 +52,7 @@ snippet...
access-key = ${AWS_ACCESS_KEY}
secret-key = ${AWS_SECRET_ACCESS_KEY}
signing-region = ${SQS_REGION}
service-endpoint = ${SQS_ENDPOINT}
service-endpoint = ${SQS_SERVICE_ENDPOINT}
queue-name = ${SQS_QUEUE_NAME}
}
```

View File

@ -251,7 +251,7 @@ portal.vinyldns.backend.url = "http://vinyldns-api:9000"
portal.test_login = false
# configuration for the users and groups store
data-stores = ["dynamodb", "mysql"]
data-stores = ["mysql"]
mysql {
class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider"
@ -284,37 +284,6 @@ mysql {
}
}
dynamodb {
class-name = "vinyldns.dynamodb.repository.DynamoDBDataStoreProvider"
settings {
key = "x"
secret = "x"
endpoint = "http://vinyldns-dynamodb:8000"
region = "us-east-1"
}
repositories {
user-change {
table-name = "userChangeTest"
provisioned-reads = 30
provisioned-writes = 20
}
}
LDAP {
user="test"
password="test"
domain="test"
searchBase = [{organization = "someDomain", domainName = "DC=test,DC=test,DC=com"}, {organization = "anotherDomain", domainName = "DC=test,DC=com"}]
context {
initialContextFactory = "com.sun.jndi.ldap.LdapCtxFactory"
securityAuthentication = "simple"
providerUrl = "ldaps://somedomain.com:9999"
}
}
play.filters.enabled += "play.filters.csrf.CSRFFilter"

View File

@ -1,5 +1,7 @@
mysql {
class-name = "vinyldns.mysql.repository.MySqlDataStoreProvider"
endpoint = "localhost:19002"
endpoint = ${?MYSQL_ENDPOINT}
settings {
# JDBC Settings, these are all values in scalikejdbc-config, not our own
@ -7,8 +9,8 @@ mysql {
# assumes a docker or mysql instance running locally
name = "vinyldns2"
driver = "org.mariadb.jdbc.Driver"
migration-url = "jdbc:mariadb://localhost:19002/"
url = "jdbc:mariadb://localhost:19002/vinyldns2"
migration-url = "jdbc:mariadb://"${mysql.endpoint}"/"
url = "jdbc:mariadb://"${mysql.endpoint}"/vinyldns2"
user = "root"
password = "pass"
@ -43,8 +45,8 @@ queue {
settings = {
name = "vinyldns2"
driver = "org.mariadb.jdbc.Driver"
migration-url = "jdbc:mariadb://localhost:19002/?user=root&password=pass"
url = "jdbc:mariadb://localhost:19002/vinyldns2?user=root&password=pass"
migration-url = "jdbc:mariadb://"${mysql.endpoint}"/?user=root&password=pass"
url = "jdbc:mariadb://"${mysql.endpoint}"/vinyldns2?user=root&password=pass"
user = "root"
password = "pass"

View File

@ -6,7 +6,7 @@
-- Ex: "jdbc:h2:mem:vinyldns;MODE=MYSQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE;INIT=RUNSCRIPT FROM 'classpath:test/ddl.sql'"
--
CREATE TABLE batch_change
CREATE TABLE IF NOT EXISTS batch_change
(
id char(36) not null primary key,
user_id char(36) not null,
@ -22,16 +22,16 @@ CREATE TABLE batch_change
cancelled_timestamp datetime null
);
create index batch_change_approval_status_index
CREATE INDEX IF NOT EXISTS batch_change_approval_status_index
on batch_change (approval_status);
create index batch_change_user_id_created_time_index
CREATE INDEX IF NOT EXISTS batch_change_user_id_created_time_index
on batch_change (user_id, created_time);
create index batch_change_user_id_index
CREATE INDEX IF NOT EXISTS batch_change_user_id_index
on batch_change (user_id);
create table group_change
CREATE TABLE IF NOT EXISTS group_change
(
id char(36) not null primary key,
group_id char(36) not null,
@ -39,10 +39,10 @@ create table group_change
data blob not null
);
create index group_change_group_id_index
CREATE INDEX IF NOT EXISTS group_change_group_id_index
on group_change (group_id);
create table `groups`
CREATE TABLE IF NOT EXISTS `groups`
(
id char(36) not null primary key,
name varchar(256) not null,
@ -52,10 +52,10 @@ create table `groups`
email varchar(256) not null
);
create index groups_name_index
CREATE INDEX IF NOT EXISTS groups_name_index
on `groups` (name);
create table membership
CREATE TABLE IF NOT EXISTS membership
(
user_id char(36) not null,
group_id char(36) not null,
@ -63,7 +63,7 @@ create table membership
primary key (user_id, group_id)
);
create table message_queue
CREATE TABLE IF NOT EXISTS message_queue
(
id char(36) not null primary key,
message_type tinyint null,
@ -75,16 +75,16 @@ create table message_queue
attempts int default 0 not null
);
create index message_queue_inflight_index
CREATE INDEX IF NOT EXISTS message_queue_inflight_index
on message_queue (in_flight);
create index message_queue_timeout_index
CREATE INDEX IF NOT EXISTS message_queue_timeout_index
on message_queue (timeout_seconds);
create index message_queue_updated_index
CREATE INDEX IF NOT EXISTS message_queue_updated_index
on message_queue (updated);
create table record_change
CREATE TABLE IF NOT EXISTS record_change
(
id char(36) not null primary key,
zone_id char(36) not null,
@ -93,13 +93,13 @@ create table record_change
data blob not null
);
create index record_change_created_index
CREATE INDEX IF NOT EXISTS record_change_created_index
on record_change (created);
create index record_change_zone_id_index
CREATE INDEX IF NOT EXISTS record_change_zone_id_index
on record_change (zone_id);
create table recordset
CREATE TABLE IF NOT EXISTS recordset
(
id char(36) not null primary key,
zone_id char(36) not null,
@ -112,16 +112,16 @@ create table recordset
unique (zone_id, name, type)
);
create index recordset_fqdn_index
CREATE INDEX IF NOT EXISTS recordset_fqdn_index
on recordset (fqdn);
create index recordset_owner_group_id_index
CREATE INDEX IF NOT EXISTS recordset_owner_group_id_index
on recordset (owner_group_id);
create index recordset_type_index
CREATE INDEX IF NOT EXISTS recordset_type_index
on recordset (type);
create table single_change
CREATE TABLE IF NOT EXISTS single_change
(
id char(36) not null primary key,
seq_num smallint not null,
@ -138,13 +138,13 @@ create table single_change
on delete cascade
);
create index single_change_batch_change_id_index
CREATE INDEX IF NOT EXISTS single_change_batch_change_id_index
on single_change (batch_change_id);
create index single_change_record_set_change_id_index
CREATE INDEX IF NOT EXISTS single_change_record_set_change_id_index
on single_change (record_set_change_id);
create table stats
CREATE TABLE IF NOT EXISTS stats
(
id bigint auto_increment primary key,
name varchar(255) not null,
@ -152,13 +152,13 @@ create table stats
created datetime not null
);
create index stats_name_created_index
CREATE INDEX IF NOT EXISTS stats_name_created_index
on stats (name, created);
create index stats_name_index
CREATE INDEX IF NOT EXISTS stats_name_index
on stats (name);
create table task
CREATE TABLE IF NOT EXISTS task
(
name varchar(255) not null primary key,
in_flight bit not null,
@ -166,7 +166,7 @@ create table task
updated datetime null
);
create table user
CREATE TABLE IF NOT EXISTS user
(
id char(36) not null primary key,
user_name varchar(256) not null,
@ -174,13 +174,13 @@ create table user
data blob not null
);
create index user_access_key_index
CREATE INDEX IF NOT EXISTS user_access_key_index
on user (access_key);
create index user_user_name_index
CREATE INDEX IF NOT EXISTS user_user_name_index
on user (user_name);
create table user_change
CREATE TABLE IF NOT EXISTS user_change
(
change_id char(36) not null primary key,
user_id char(36) not null,
@ -188,7 +188,7 @@ create table user_change
created_timestamp bigint(13) not null
);
create table zone
CREATE TABLE IF NOT EXISTS zone
(
id char(36) not null primary key,
name varchar(256) not null,
@ -198,13 +198,13 @@ create table zone
unique (name)
);
create index zone_admin_group_id_index
CREATE INDEX IF NOT EXISTS zone_admin_group_id_index
on zone (admin_group_id);
create index zone_name_index
CREATE INDEX IF NOT EXISTS zone_name_index
on zone (name);
create table zone_access
CREATE TABLE IF NOT EXISTS zone_access
(
accessor_id char(36) not null,
zone_id char(36) not null,
@ -214,13 +214,13 @@ create table zone_access
on delete cascade
);
create index zone_access_accessor_id_index
CREATE INDEX IF NOT EXISTS zone_access_accessor_id_index
on zone_access (accessor_id);
create index zone_access_zone_id_index
CREATE INDEX IF NOT EXISTS zone_access_zone_id_index
on zone_access (zone_id);
create table zone_change
CREATE TABLE IF NOT EXISTS zone_change
(
change_id char(36) not null primary key,
zone_id char(36) not null,
@ -228,11 +228,13 @@ create table zone_change
created_timestamp bigint(13) not null
);
create index zone_change_created_timestamp_index
CREATE INDEX IF NOT EXISTS zone_change_created_timestamp_index
on zone_change (created_timestamp);
create index zone_change_zone_id_index
CREATE INDEX IF NOT EXISTS zone_change_zone_id_index
on zone_change (zone_id);
DELETE FROM task WHERE name = 'user_sync';
INSERT IGNORE INTO task(name, in_flight, created, updated)
VALUES ('user_sync', 0, NOW(), NULL);

View File

@ -29,8 +29,8 @@ object MySqlConnector {
private val logger = LoggerFactory.getLogger("MySqlConnector")
def runDBMigrations(config: MySqlConnectionConfig): IO[Unit] =
// We can skip migrations for h2, we'll use the test/ddl.sql for initializing
// that for testing
// We can skip migrations for h2, we'll use the test/ddl.sql for initializing
// that for testing
if (config.driver.contains("h2")) IO.unit
else {
val migrationConnectionSettings = MySqlDataSourceSettings(
@ -90,14 +90,21 @@ object MySqlConnector {
def retry[T](times: Int, delayMs: Int)(op: => T) =
Iterator
.range(0, times)
.map(_ => Try(op))
.flatMap {
case Success(t) => Some(t)
case Failure(_) =>
logger.warn("failed to startup database connection, retrying..")
Thread.sleep(delayMs)
None
}
.map(index => (index, Try(op)))
.flatMap(x => {
val (currentIndex, result) = x
result match {
case Success(t) => Some(t)
case Failure(e) =>
logger.warn("failed to startup database connection, retrying..")
// Hard abort if we exhaust retries
if (currentIndex >= times - 1) {
throw e
}
Thread.sleep(delayMs)
None
}
})
.toSeq
.head

View File

@ -1,23 +1,62 @@
http.port = 9001
LDAP {
# For OpenLDAP, this would be a full DN to the admin for LDAP / user that can see all users
user = "cn=admin,dc=planetexpress,dc=com"
crypto {
type = "vinyldns.core.crypto.JavaCrypto"
secret = "8B06A7F3BC8A2497736F1916A123AA40E88217BE9264D8872597EF7A6E5DCE61"
}
# Password for the admin account
password = "GoodNewsEveryone"
data-stores = ["mysql"]
data-stores = ${?DATA_STORES}
# Keep this as an empty string for OpenLDAP
domain = ""
mysql {
repositories {
user {
}
task {
}
user-change {
}
# This will be the name of the LDAP field that carries the user's login id (what they enter in the username in login form)
userNameAttribute = "uid"
# For organization, leave empty for this demo, the domainName is what matters, and that is the LDAP structure
# to search for users that require login
searchBase = [
{organization = "", domainName = "ou=people,dc=planetexpress,dc=com"},
]
context {
initialContextFactory = "com.sun.jndi.ldap.LdapCtxFactory"
initialContextFactory = ${?LDAP_INITIAL_CONTEXT_CLASS}
securityAuthentication = "simple"
securityAuthentication = ${?LDAP_SECURITY_AUTH}
# Note: The following assumes a purely docker setup, using container_name = vinyldns-ldap
providerUrl = "ldap://vinyldns-ldap:19004"
providerUrl = ${?LDAP_PROVIDER_URL}
}
# This is only needed if keeping vinyldns user store in sync with ldap (to auto lock out users who left your
# company for example)
user-sync {
enabled = false
enabled = ${?USER_SYNC_ENABLED}
hours-polling-interval = 1
hours-polling-interval = ${?USER_SYNC_POLL_INTERVAL}
}
}
// Local.conf has files specific to your environment, for example your own LDAP settings
# Note: This MUST match the API or strange errors will ensue, NoOpCrypto should not be used for production
crypto {
type = "vinyldns.core.crypto.NoOpCrypto"
type = ${?CRYPTO_TYPE}
secret = ${?CRYPTO_SECRET}
}
http.port = 9001
http.port = ${?PORTAL_PORT}
data-stores = ["mysql"]
# Must be true to manage shared zones through the portal
shared-display-enabled = true
shared-display-enabled = ${?SHARED_ZONES_ENABLED}
# You generate this yourself following https://www.playframework.com/documentation/2.7.x/ApplicationSecret
play.http.secret.key = "changeme"
play.http.secret.key = ${?PLAY_HTTP_SECRET_KEY}
# You can provide configuration overrides via local.conf if you don't want to replace everything in
# this configuration file
include "local.conf"

View File

@ -30,7 +30,7 @@ import vinyldns.core.domain.{Fqdn, record}
import vinyldns.core.domain.record.{RecordSet, RecordType}
class Route53IntegrationSpec
extends AnyWordSpec
extends AnyWordSpec
with BeforeAndAfterAll
with BeforeAndAfterEach
with Matchers {
@ -52,7 +52,7 @@ class Route53IntegrationSpec
"test",
Option("access"),
Option("secret"),
"http://127.0.0.1:19003",
sys.env.getOrElse("R53_SERVICE_ENDPOINT", "http://localhost:19003"),
"us-east-1"
)
)

View File

@ -8,6 +8,7 @@ sqs {
secret-key = "test"
signing-region = "us-east-1"
service-endpoint = "http://localhost:19003/"
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
queue-name = "sqs-override-name"
}
}

View File

@ -15,6 +15,7 @@
*/
package vinyldns.sqs.queue
import com.typesafe.config.ConfigFactory
import org.scalatest.matchers.should.Matchers
import org.scalatest.wordspec.AnyWordSpec
@ -24,50 +25,53 @@ import pureconfig._
import pureconfig.generic.auto._
class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
val undertest = new SqsMessageQueueProvider()
val underTest = new SqsMessageQueueProvider()
private val sqsEndpoint = sys.env.getOrElse("SQS_SERVICE_ENDPOINT", "http://localhost:19003")
"load" should {
"fail if a required setting is not provided" in {
val badConfig =
ConfigFactory.parseString("""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| service-endpoint = "http://localhost:19003/"
| queue-name = "queue-name"
| }
| """.stripMargin)
ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| service-endpoint = "$sqsEndpoint"
| queue-name = "queue-name"
| }
| """.stripMargin)
val badSettings = ConfigSource.fromConfig(badConfig).loadOrThrow[MessageQueueConfig]
a[pureconfig.error.ConfigReaderException[MessageQueueConfig]] should be thrownBy undertest
a[pureconfig.error.ConfigReaderException[MessageQueueConfig]] should be thrownBy underTest
.load(badSettings)
.unsafeRunSync()
}
"create the queue if the queue is non-existent" in {
val nonExistentQueueConfig =
ConfigFactory.parseString("""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/"
| queue-name = "new-queue"
| }
| """.stripMargin)
ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "$sqsEndpoint"
| queue-name = "new-queue"
| }
| """.stripMargin)
val messageConfig =
ConfigSource.fromConfig(nonExistentQueueConfig).loadOrThrow[MessageQueueConfig]
val messageQueue = undertest.load(messageConfig).unsafeRunSync()
val messageQueue = underTest.load(messageConfig).unsafeRunSync()
noException should be thrownBy messageQueue
.asInstanceOf[SqsMessageQueue]
@ -77,65 +81,68 @@ class SqsMessageQueueProviderIntegrationSpec extends AnyWordSpec with Matchers {
"fail with InvalidQueueName if an invalid queue name is given" in {
val invalidQueueNameConfig =
ConfigFactory.parseString("""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/"
| queue-name = "bad*queue*name"
| }
| """.stripMargin)
ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "$sqsEndpoint"
| queue-name = "bad*queue*name"
| }
| """.stripMargin)
val messageConfig =
ConfigSource.fromConfig(invalidQueueNameConfig).loadOrThrow[MessageQueueConfig]
assertThrows[InvalidQueueName](undertest.load(messageConfig).unsafeRunSync())
assertThrows[InvalidQueueName](underTest.load(messageConfig).unsafeRunSync())
}
"fail with InvalidQueueName if a FIFO queue is specified" in {
val fifoQueueName =
ConfigFactory.parseString("""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/"
| queue-name = "queue.fifo"
| }
| """.stripMargin)
ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "$sqsEndpoint"
| queue-name = "queue.fifo"
| }
| """.stripMargin)
val messageConfig = ConfigSource.fromConfig(fifoQueueName).loadOrThrow[MessageQueueConfig]
assertThrows[InvalidQueueName](undertest.load(messageConfig).unsafeRunSync())
assertThrows[InvalidQueueName](underTest.load(messageConfig).unsafeRunSync())
}
}
"MessageQueueLoader" should {
"invoke SQS provider properly" in {
val nonExistentQueueConfig =
ConfigFactory.parseString("""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "http://localhost:19003/"
| queue-name = "new-queue"
| }
| """.stripMargin)
ConfigFactory.parseString(
s"""
| class-name = "vinyldns.sqs.queue.SqsMessageQueueProvider"
| polling-interval = 250.millis
| messages-per-poll = 10
| max-retries = 100
|
| settings {
| access-key = "x"
| secret-key = "x"
| signing-region = "us-east-1"
| service-endpoint = "$sqsEndpoint"
| queue-name = "new-queue"
| }
| """.stripMargin)
val messageConfig =
ConfigSource.fromConfig(nonExistentQueueConfig).loadOrThrow[MessageQueueConfig]

View File

@ -11,8 +11,9 @@ TEST_LOGIN=false
# API Settings
REST_PORT=9000
SQS_ENDPOINT=http://vinyldns-integration:19003
SQS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
R53_SERVICE_ENDPOINT=http://vinyldns-integration:19003
MYSQL_ENDPOINT=vinyldns-integration:19002
DEFAULT_DNS_ADDRESS=vinyldns-integration:19001

View File

@ -85,7 +85,7 @@ vinyldns {
# Endpoint to access queue
service-endpoint = "http://localhost:19003/"
service-endpoint = ${?SQS_ENDPOINT}
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"

View File

@ -0,0 +1,27 @@
# Overrides that can be used when dependent services are running in the vinyldns-integration host
# and 'sbt' or tests are running in another container. More importantly, when these services are
# not accessible via "localhost"
#
# Any docker container that wishes to to use these, can simply specify this file using the `--env-file`
# argument.
# General settings
VINYLDNS_API_URL=http://vinyldns-integration:9000
VINYLDNS_PORTAL_URL=http://vinyldns-integration:9001
# Portal settings
VINYLDNS_BACKEND_URL=http://vinyldns-integration:9000
LDAP_PROVIDER_URL=ldap://vinyldns-integration:19004
# API Settings
SQS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
SNS_SERVICE_ENDPOINT=http://vinyldns-integration:19003
R53_SERVICE_ENDPOINT=http://vinyldns-integration:19003
MYSQL_ENDPOINT=vinyldns-integration:19002
DEFAULT_DNS_ADDRESS=vinyldns-integration:19001
JDBC_DRIVER=org.mariadb.jdbc.Driver
JDBC_USER=root
JDBC_PASSWORD=pass
JDBC_URL=jdbc:mariadb://vinyldns-integration:19002/vinyldns?user=${JDBC_USER}&password=${JDBC_PASSWORD}
JDBC_MIGRATION_URL=jdbc:mariadb://vinyldns-integration:19002/?user=${JDBC_USER}&password=${JDBC_PASSWORD}

View File

@ -38,14 +38,16 @@ build:
run:
@set -euo pipefail
docker network create --driver bridge vinyldns_net &> /dev/null || true
USE_TTY="" && test -t 1 && USE_TTY="-t"
docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) $(IMAGE_NAME) $(ARG_SEPARATOR) $(WITH_ARGS)
docker run -i $${USE_TTY} --rm --network vinyldns_net $(DOCKER_PARAMS) $(IMAGE_NAME) $(ARG_SEPARATOR) $(WITH_ARGS)
run-bg:
@set -euo pipefail
docker stop $(IMAGE_NAME) &> /dev/null || true
docker network create --driver bridge vinyldns_net &> /dev/null || true
USE_TTY="" && test -t 1 && USE_TTY="-t"
docker run -d $${USE_TTY} --name $(IMAGE_NAME) --rm $(DOCKER_PARAMS) -e RUN_SERVICES="deps-only tail-logs" -p 19001-19003:19001-19003 -p 19001:19001/udp $(IMAGE_NAME)
docker run -d $${USE_TTY} --name $(IMAGE_NAME) --rm --network vinyldns_net $(DOCKER_PARAMS) -e RUN_SERVICES="deps-only tail-logs" -p 19001-19003:19001-19003 -p 19001:19001/udp $(IMAGE_NAME)
stop-bg:
@set -euo pipefail
@ -53,8 +55,9 @@ stop-bg:
run-local:
@set -euo pipefail
docker network create --driver bridge vinyldns_net &> /dev/null || true
USE_TTY="" && test -t 1 && USE_TTY="-t"
docker run -i $${USE_TTY} --rm $(DOCKER_PARAMS) -v "$(ROOT_DIR)/../../..:/build" $(IMAGE_NAME) -- $(WITH_ARGS)
docker run -i $${USE_TTY} --rm --network vinyldns_net $(DOCKER_PARAMS) -v "$(ROOT_DIR)/../../..:/build" $(IMAGE_NAME) -- $(WITH_ARGS)
clean-containers:
@set -euo pipefail

View File

@ -114,7 +114,7 @@ vinyldns {
# Endpoint to access queue
service-endpoint = "http://localhost:19003/"
service-endpoint = ${?SQS_ENDPOINT}
service-endpoint = ${?SQS_SERVICE_ENDPOINT}
# Queue name. Should be used in conjunction with service endpoint, rather than using a queue url which is subject to change.
queue-name = "vinyldns"