diff --git a/DNS-Shotgun-integration-into-Gitlab-CI.md b/DNS-Shotgun-integration-into-Gitlab-CI.md new file mode 100644 index 0000000..db4e989 --- /dev/null +++ b/DNS-Shotgun-integration-into-Gitlab-CI.md @@ -0,0 +1,21 @@ +Technical notes about DNS Shotgun integration into Gitlab CI +======== + +High-level overview, from top to bottom +---- +1. User starts Shotgun test job in Gitlab CI with tag `linux-benchmarking` +2. Gitlab directs job to dedicated Docker executor on VM running inside AWS. + - Runner VM needs couple [configuration tricks](https://gitlab.isc.org/isc-private/devops/-/merge_requests/8) to get IPv6 to work inside Docker container running on AWS VM +3. Docker executor starts `.gitlab-ci.yaml` script inside dedicated Docker image [shotgun-controller](https://gitlab.isc.org/isc-projects/images/-/merge_requests/114) +4. [Script `shotgun_aws.py`](https://gitlab.isc.org/isc-private/bind-qa/-/merge_requests/35) creates two ephemeral VMs in AWS, dedicated for this test: + - To do that, the script needs AWS permissions to manage VMs and related resources. Runner machine is associated with special AWS ACL `arn:aws:iam::766250944489:role/al2-amd64-bind9-resolver-benchmarking-IAMRole-OKGJY309ABZR`. + - VMs use AMI (VM image) [dedicated for DNS Shotgun tests](https://gitlab.isc.org/isc-projects/images/-/merge_requests/113) + - To avoid hardcoding values into `shotgun_aws.py`, the script uses AWS Launch Template (ID `lt-0161f30b78633fdb2`) which can be modified in AWS console + - New VMs are tagged with timestamp in `isc:remove_after` tag, which denotes deadline after which the job has to be finished and all resources in AWS can be deleted. This is intended as guard against unlimited spending when Gitlab CI job is cancelled before it finishes teardown phase. + - Cleanup script [`cleanup_ephemeral.py`](https://gitlab.isc.org/isc-private/bind-qa/-/merge_requests/35) is run from cron job on Gitlab CI runner machine. +5. When VMs are ready, `shotgun_aws.py` executes [Ansible playbook](https://gitlab.nic.cz/knot/resolver-benchmarking/) which orchestrates the test on the two VMs. +6. The Ansible playbook connects to test VMs using SSH and runs DNS Shotgun on one machine and resolver under test on the other machine. + - VMs act as Docker hosts, i.e. Shotgun and resolver run inside Docker containers + - Docker networking is disabled using `--network=host` + - VM running DNS Shotgun has extra partition with PCAPs (AWS snapshot ID `snap-0ae93969448ffb83f`) +7. When test is finished, Ansible playbook gathers test results and stores them inside Docker container executed directly by Gitlab CI. \ No newline at end of file