Merge pull request #33 from nimaeskandary/portal-docker

Add portal docker image

.gitignore

@@ -24,3 +24,4 @@ release.version
 .ensime
 .ensime_cache
 package-lock.json
+*trustStore.jks

DEVELOPER_GUIDE.md

@@ -106,11 +106,26 @@ Run `protoc --version`, if it is not 2.6.1, then
 Be sure to install the latest version of [docker](https://docs.docker.com/).  You must have docker running in order to work with VinylDNS on your machine.
 Be sure to start it up if it is not running before moving further.
 
-### How to use the Docker Image
-#### Starting a vinyldns-api server instance
-VinylDNS depends on several dependencies including mysql, sqs, dynamodb and a DNS server.  These can be passed in as
+#### Starting a vinyldns installation locally in docker
+Running `./bin/docker-up-vinyldns.sh` will spin up the production docker images of the vinyldns-api and vinyldns-portal.
+This will startup all the dependencies as well as the api and portal servers.  
+It will then ping the api on `http://localhost:9000` and the portal on `http://localhost:9001` and notify you if either failed to start.
+The portal can be viewed in a browser at `http://localhost:9001`
+
+Alternatively, you can manually run docker-compose with this config `docker/docker-compose-build.yml`.
+From the root directory run `docker-compose -f ./docker/docker-compose-build.yml up -d`
+
+To stop the local setup, run `./bin/stop-all-docker-containers.sh` from the project root.
+
+> Warning: the `./bin/stop-all-docker-containers.sh` will stop and remove all local docker containers
+
+### Configuration for the vinyldns-api image
+VinylDNS depends on several dependencies including mysql, sqs, dynamodb and a DNS server. These can be passed in as
 environment variables, or you can override the config file with your own settings.
 
+By default, the api image is configured to run in a docker compose environment locally. To run in a production environment,
+you would have to configure the portal appropriately. 
+
 #### Environment variables
 1. `MYSQL_ADDRESS` - the IP address of the mysql server; defaults to `vinyldns-mysql` assuming a docker compose setup
 1. `MYSQL_PORT` - the port of the mysql server; defaults to 3306
@@ -126,21 +141,30 @@ variables.
 #### Ports
 vinyldns only exposes port 9000 for HTTP access to all endpoints
 
-#### Starting a vinyldns installation locally in docker
-There is a handy docker-compose file for spinning up the production docker image on your local under `docker/docker-compose-build.yml`
+### Configuration for the vinyldns-portal image
+Like the api image, the portal image is configured to run in a docker compose environment locally. To run in a production environment,
+you would have to configure the portal appropriately using these settings. 
 
-From the root directory run...
+#### Volume mounts
+* `/opt/docker/lib_extra` - place here additional jar files that need to be loaded into the classpath when the application starts up.
+This is used for "plugins" that are proprietary or not part of the standard build.  All jar files here will be placed on the class path.
+* `/opt/docker/conf/application.conf` - to override default configuration settings
+* `/opt/docker/conf/application.ini` - to pass additional JVM options 
+* `/opt/docker/conf/trustStore.jks` - to make available a custom trustStore, which has to be set in `/opt/docker/conf/application.ini` as `-Djavax.net.ssl.trustStore=/opt/docker/conf/trustStore.jks`
 
-```
-> docker-compose -f ./docker/docker-compose-build.yml up -d
-```
+#### Custom LDAP config
+In `docker/portal/application.conf` there is a switch for `portal.test_login = true`. This is set by default so 
+developers can login to the portal with username=testuser and password=testpassword. Custom LDAP settings will have to 
+be set in `docker/portal/application.conf`
 
-This will startup all the dependencies as well as the api server.  Once the api server is running, you can verify it is
-up by running the following `curl -v http://localhost:9000/status`
+#### Configuring a custom Java trustStore
+To add a custom Java trustStore, say for LDAP certs, add the trustStore to `docker/portal/trustStore.jks`. Then
+add `-Djavax.net.ssl.trustStore=/opt/docker/conf/trustStore.jks` to `docker/portal/application.ini`.  
 
-To stop the local setup, run `./bin/stop-all-docker-containers.sh` from the project root.
+#### Additional JVM parameters
+Additional JVM parameters can be added to `docker/portal/application.ini`
 
-#### Validating everything
+### Validating everything
 VinylDNS comes with a build script `./build.sh` that validates, verifies, and runs functional tests.  Note: This
 takes a while to run, and typically is only necessary if you want to simulate the same process that runs on the build
 servers
@@ -224,8 +248,10 @@ server.  If you need to add or modify the test DNS zone files, you can find them
 `docker/bind9/zones`
 
 ## Handy Scripts
-### Start up a complete local API server
-`bin/docker-up-api-server.sh` - this will build vinyl (if not built) and then start up an api server and all dependencies
+### Start up complete local API and Portal servers
+`bin/docker-up-vinyldns.sh` - this will start up the `vinyldns/api:latest` and `vinyldns/portal:latest` images from docker hub
+
+> Note: to start up images with local changes, run `sbt ;project api; docker:publishLocal; project portal; docker:publishLocal`
 
 The following ports and services are available:
 

bin/docker-up-vinyldns.sh

@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+######################################################################
+# Copies the contents of `docker` into target/scala-2.12
+# to start up dependent services via docker compose.  Once
+# dependent services are started up, the fat jar built by sbt assembly
+# is loaded into a docker container.  The api will be available
+## by default on port 9000 and the portal will be on port 9001
+######################################################################
+
+DIR=$( cd $(dirname $0) ; pwd -P )
+
+echo "Starting portal server and all dependencies in the background..."
+docker-compose -f $DIR/../docker/docker-compose-build.yml up -d
+
+VINYL_URL="http://localhost:9000"
+echo "Waiting for API to be ready at ${VINYL_URL} ..."
+DATA=""
+RETRY=40
+while [ $RETRY -gt 0 ]
+do
+    DATA=$(wget -O - -q -t 1 "${VINYL_URL}/ping")
+    if [ $? -eq 0 ]
+    then
+        echo "Succeeded in connecting to VINYL API!"
+        break
+    else
+        echo "Retrying Again" >&2
+
+        let RETRY-=1
+        sleep 1
+
+        if [ $RETRY -eq 0 ]
+        then
+          echo "Exceeded retries waiting for VINYL API to be ready, failing"
+          exit 1
+        fi
+    fi
+done
+
+VINYL_URL="http://localhost:9001"
+echo "Waiting for PORTAL to be ready at ${VINYL_URL} ..."
+DATA=""
+RETRY=40
+while [ $RETRY -gt 0 ]
+do
+    DATA=$(wget -O - -q -t 1 "${VINYL_URL}")
+    if [ $? -eq 0 ]
+    then
+        echo "Succeeded in connecting to VINYL PORTAL!"
+        break
+    else
+        echo "Retrying Again" >&2
+
+        let RETRY-=1
+        sleep 1
+
+        if [ $RETRY -eq 0 ]
+        then
+          echo "Exceeded retries waiting for VINYL PORTAL to be ready, failing"
+          exit 1
+        fi
+    fi
+done

build.sbt

@@ -108,6 +108,7 @@ lazy val apiDockerSettings = Seq(
   dockerBaseImage := "openjdk:8u171-jdk",
   dockerUsername := Some("vinyldns"),
   packageName in Docker := "api",
+  dockerUpdateLatest := true,
   dockerExposedPorts := Seq(9000),
   dockerEntrypoint := Seq("/opt/docker/bin/boot"),
   dockerExposedVolumes := Seq("/opt/docker/lib_extra"), // mount extra libs to the classpath
@@ -130,6 +131,24 @@ lazy val apiDockerSettings = Seq(
   composeFile := baseDirectory.value.getAbsolutePath + "/../../docker/docker-compose.yml"
 )
 
+lazy val portalDockerSettings = Seq(
+  dockerBaseImage := "openjdk:8u171-jdk",
+  dockerUsername := Some("vinyldns"),
+  packageName in Docker := "portal",
+  dockerUpdateLatest := true,
+  dockerExposedPorts := Seq(9001),
+  dockerExposedVolumes := Seq("/opt/docker/lib_extra"), // mount extra libs to the classpath
+  dockerExposedVolumes := Seq("/opt/docker/conf"), // mount extra config to the classpath
+
+  // add extra libs to class path via mount
+  scriptClasspath in bashScriptDefines ~= (cp => cp :+ "/opt/docker/lib_extra/*"),
+
+  // adds config file to mount
+  bashScriptExtraDefines += """addJava "-Dconfig.file=/opt/docker/conf/application.conf"""",
+  bashScriptExtraDefines += """addJava "-Dlogback.configurationFile=/opt/docker/conf/logback.xml"""",
+  credentials in Docker := Seq(Credentials(Path.userHome / ".iv2" / ".dockerCredentials"))
+)
+
 lazy val noPublishSettings = Seq(
   publish := {},
   publishLocal := {},
@@ -142,6 +161,12 @@ lazy val apiPublishSettings = Seq(
   publish := (publish in Docker).value
 )
 
+lazy val portalPublishSettings = Seq(
+  publishArtifact := false,
+  publishLocal := (publishLocal in Docker).value,
+  publish := (publish in Docker).value
+)
+
 lazy val pbSettings = Seq(
   version in ProtobufConfig := "2.6.1"
 )
@@ -213,7 +238,8 @@ val createJsHeaders = TaskKey[Unit]("createJsHeaders", "Runs script to prepend A
 lazy val portal = (project in file("modules/portal")).enablePlugins(PlayScala, AutomateHeaderPlugin)
   .settings(sharedSettings)
   .settings(testSettings)
-  .settings(noPublishSettings)
+  .settings(portalPublishSettings)
+  .settings(portalDockerSettings)
   .settings(
     name := "portal",
     libraryDependencies ++= portalDependencies,

docker/docker-compose-build.yml

@@ -35,7 +35,7 @@ services:
       - ./elasticmq/custom.conf:/etc/elasticmq/elasticmq.conf
 
   api:
-    image: vinyldns/api:0.1 # the version of the docker container we want to pull
+    image: "vinyldns/api:latest"
     environment:
       - REST_PORT=9000
     container_name: "vinyldns-api"
@@ -46,3 +46,14 @@ services:
       - bind9
       - elasticmq
       - dynamodb
+
+  portal:
+    image: "vinyldns/portal:latest"
+    ports:
+      - "9001:9001"
+    container_name: "vinyldns-portal"
+    volumes:
+      - ./portal/application.conf:/opt/docker/conf/application.conf
+      - ./portal/application.ini:/opt/docker/conf/application.ini
+    depends_on:
+      - api

docker/portal/application.conf

@@ -0,0 +1,82 @@
+# This is the main configuration file for the application.
+# ~~~~~
+
+# Secret key
+# ~~~~~
+# The secret key is used to secure cryptographics functions.
+#
+# This must be changed for production, but we recommend not changing it in this file.
+#
+# See http://www.playframework.com/documentation/latest/ApplicationSecret for more details.
+play.http.secret.key = "vinyldnsportal-change-this-for-production"
+
+# The application languages
+# ~~~~~
+play.i18n.langs = [ "en" ]
+
+portal.dynamo_delay = 1100
+portal.vinyldns.backend.url = "http://vinyldns-api:9000"
+portal.test_login = true
+
+# configuration for the users and groups store
+dynamo {
+  key = "akid goes here"
+  secret = "secret key goes here"
+  endpoint = "http://vinyldns-dynamodb:8000"
+  test_datastore = false
+}
+
+users {
+  dummy = false
+  tablename = "users"
+  provisionedReadThroughput = 100
+  provisionedWriteThroughput = 100
+}
+
+changelog {
+  dummy = false
+  tablename = "usersAndGroupChanges"
+  provisionedReadThroughput = 100
+  provisionedWriteThroughput = 100
+}
+
+LDAP {
+  user="test"
+  password="test"
+  domain="test"
+
+  searchBase = [{organization = "someDomain", domainName = "DC=test,DC=test,DC=com"}, {organization = "anotherDomain", domainName = "DC=test,DC=com"}]
+
+  context {
+    initialContextFactory = "com.sun.jndi.ldap.LdapCtxFactory"
+    securityAuthentication = "simple"
+    providerUrl = "ldaps://somedomain.com:9999"
+  }
+
+}
+
+play.filters.enabled += "play.filters.csrf.CSRFFilter"
+
+# Expire session after 10 hours
+play.http.session.maxAge = 10h
+
+# session secure should be false in order to run properly locally, this is set properly on deployment
+play.http.session.secure = false
+play.http.session.httpOnly = true
+
+# use no-op by default
+crypto {
+  type = "vinyldns.core.crypto.NoOpCrypto"
+}
+
+http.port=9001
+
+links = [
+  {
+    displayOnSidebar = true
+    displayOnLoginScreen = true
+    title = "API Documentation"
+    href = "http://vinyldns.io"
+    icon = "fa fa-file-text-o"
+  }
+]

docker/portal/application.ini

@@ -0,0 +1,3 @@
+# uncomment to set custom trustStore
+# don't forget to mount trustStore to docker image
+#-Djavax.net.ssl.trustStore=/opt/docker/conf/trustStore.jks

modules/api/src/universal/conf/application.conf

@@ -35,7 +35,7 @@ vinyldns {
     local-mode = true
     default {
       driver = "org.mariadb.jdbc.Driver"
-      migrationUrl = "jdbc:mariadb://localhost:3306/?user=root&password=pass"
+      migrationUrl = "jdbc:mariadb://vinyldns-mysql:3306/?user=root&password=pass"
       url = "jdbc:mariadb://vinyldns-mysql:3306/vinyldns?user=root&password=pass"
       user = "root"
       password = "pass"

modules/portal/conf/application.ini

@@ -0,0 +1,2 @@
+# set custom trustStore
+#-Djavax.net.ssl.trustStore=...