For the tutorials, I've chosen to use Debian GNU/Linux with lxc, btrfs and openvswitch and some extra git thrown at it to build simulations of complete networks. The current stable Debian Release, 10 (Buster) already contains everything you need for this.
* The advantage of using openvswitch is that we can very easily run a vlan capable switch, by just configuring ports on it as either access or trunk port with any of the vlan numbers assigned, much like you would do with a physical switch.
* Creating a git repository just outside the container root filesystems with a .gitignore that only includes specific files allows to easily store the configuration of a whole test network. For example, when destroying a complete container and cloning it from another container again, a simple git checkout is enough to put the configuration inside the container back in place.
Here's a simple schematic overview of what I mean:
The lxc host system only needs a single external network interface, for you to manage it, and probably to masquerade outgoing traffic from the test environment, using NAT. It's of course also possible to route some real address space to this box for use in the test networks, but I'm not doing so.
Here's the `/etc/network/interfaces` of my lxc host, well, almost, since I replaced the eth0 addresses with fakes:
lxcbird:/etc/network 0-# cat interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
up ip link set up dev eth0
up ip addr add 10.255.1.34/24 brd + dev eth0
up ip addr add 2001:db8:ffff::22/64 dev eth0
up ip route add default via 10.255.1.1 dev eth0
up ip route add default via 2001:db8:ffff::1 dev eth0
As you can see, I'm not a fan of the default way `network/interfaces` works in Debian. Actually, I always use manual mode and (pre-)up and (post-)down rules to set up and tear down everything. This is more convenient if you don't like magic too much, and often work with multiple addresses and extra commands on interfaces.
## Masquerading outgoing traffic
To enable masquerading outgoing traffic from the test networks, make sure you enable IP forwarding, for example by putting the following settings in the `/etc/sysctl.conf`...
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1
...and by using a few simple netfilter rules to do the NAT, like...
...which can be done for IPv4, as well as for IPv6, because NAT for IPv6 has finally been implemented. For test environments like this, it's very helpful, since we can just use documentation addresses from `2001:db8::/32` and are still able to access the outside internet if needed.
## Setting up version control
lxcbird:/var/lib/lxc 0-# git init
Initialized empty Git repository in /var/lib/lxc-bird/.git/
Now make sure your `.gitignore` looks like this, to include only very specific files from all containers:
Note that I'm using a btrfs file system on my computer, and lxc is using btrfs snapshots to clone the containers. When using a different storage backend for lxc, e.g. ext4 and overlayfs, some of the details of placement of files may vary. It is left as exercise for the reader to deal with these small differences.
In `birdbase/config`, lxc-create has put some basic configuration. The networking configuration has to be set up now, so we can test our connectivity and install some extra software. To be able to do so, I'm going to configure it with an IPv4 and IPv6 address in the range of vlan10, and point my default gateway to the lxc host system.
...and also, if you don't have apparmor stuff set up (apparently I haven't), then you can disable all of that by changing the following option to 'unconfined'. It took me a bit to figure this out, based on only a "No such file or directory" error I got:
Oh, and by the way, the lxc network script referenced is a really simple script to integrate lxc with openvswitch, which simply attaches an interface in the container to a vlan inside openvswitch based on the number after the dot. It has to be present on the host system, not in the container, and it needs the executable permission bit set.
Instead of setting the container IP address and gateway in the lxc configuration file, I prefer using network/interfaces inside the container, because we'll be using that for more complex networking anyway in the tutorials:
I hate the default of installing recommends in Debian, so I always turn that off. Generally, it's recommended to install Recommends when using Debian, so it installs other packages that are 'generally found together with these ones'. Generally, I don't really see the pattern in this, and I recommend to just try disabling Recommends to see which issues you run into, so you learn more about how related software works together. Anyway, for your minimal BIRD lxc container, we won't run into any problem doing so now.
The creation of the log file is necessary to work around a bug in the Debian packaging, that causes the logfile to be created with root as owner, and subsequent causes bird startup to fail because it cannot write to the logfile as user bird. :-(
For IP forwarding, make sure you uncomment `net.ipv4.ip_forward=1` and `net.ipv6.conf.all.forwarding=1` in sysctl.conf inside the container. Hint: editing configuration files inside a container can be done from outside the container, by looking for them in the `rootfs` folder inside the container directories.
Since we'll be doing a lot of tracerouting in the example networks, it's nice to disable icmp error rate limiting in sysctl.conf, to prevent hickups while executing quick subsequent traceroute commands:
net.ipv4.icmp_ratelimit = 0
net.ipv6.icmp.ratelimit = 0
You probably wouldn't want to do this in a production network. For more information, see [the blog post "A strange packet loss"](http://backreference.org/2012/11/16/a-strange-packet-loss/)
Before the birdbase container is ready as a template to be used for cloning other containers, let's shut it down and remove some container-specific configuration, so we won't accidentally start a new one with duplicate configuration, and, to make the diff look nicer when configuring a clone:
lxcbird:/var/lib/lxc/birdbase 0-# git commit -m "birdbase network and bird config"
[...]
Right! As you might notice, there are "end-hosts" in the drawing on top of this page, and we just configured the base container to start bird and enable IP forwarding. While this is not needed, or wanted for the end host containers, I don't really care, because it will not influence the working of the test environment, as bird has no configuration, end hosts will not attract traffic that's not for themselves. However, if you like, you can create two different containers to clone from, one for a router, and one for an end host.
