||1 year ago|
|auth1||1 year ago|
|cobalamin||1 year ago|
|coffee||1 year ago|
|dns||1 year ago|
|fs00||1 year ago|
|group_vars||1 year ago|
|localhost||1 year ago|
|1 year ago|
|outsider||1 year ago|
|phosphoric-acid||1 year ago|
|roles||1 year ago|
|uw00||1 year ago|
|.gitignore||1 year ago|
|README.md||1 year ago|
|ansible.cfg||1 year ago|
|hosts||1 year ago|
|pyceo.md||1 year ago|
The objective of this repo is to allow syscom members to create a local development environment which is reasonably close to the services which run on the CSC servers. The idea is to encourage experimentation without breaking the real services and causing outages.
This repo consists of several Ansible playbooks which will automate tasks in LXC containers. I strongly recommend creating a VM and running the containers inside the VM to avoid screwing up the network interfaces on your real computer. I am using KVM + QEMU, but VirtualBox should theoretically also work. The VM should be running some reasonably recent version of Debian or Ubuntu. 2 CPU cores, 2 GB of RAM and 30 GB of disk space should be sufficient.
Note: The localhost playbook has only been tested on Debian 10.9.0 (standard edition, i.e. no desktop). Other Debian variants should theoretically work, but have been tested. In particular, at the time of this writing, I suggest staying away from Debian 10.10.0, as it has a kernel bug which breaks LXC.
The first thing which you need to do is read the group_vars/all.yml file and see if there are any parameters which you would like to change. In particular, you may wish to set upstream_dns to the DNS server used by the VM (check /etc/resolv.conf).
The playbook localhost/main.yml should, theoretically, create all of the necessary containers and run the playbooks individually. If this doesn't work, please file a bug (make sure you test it on a "blank slate" first, i.e. destroy all existing containers).
First, install Ansible:
apt install ansible
Then run the playbook:
In each container folder, you will find instructions to setup the container manually in the README. Ignore those; they are from a time before the localhost playbook existed. The other information on the README will still be useful though.
You should only do this if the localhost playbook above is not working.
Important: You need to enable packet forwarding in the VM. Add
or uncomment the following line in
On Ubuntu, make sure you disable the default LXC bridge, as it will interfere with our own bridge:
systemctl stop lxc-net systemctl mask lxc-net
Your /etc/network/interfaces (in the VM) should look like the following:
auto enp1s0 iface enp1s0 inet dhcp auto lxcbr1 iface lxcbr1 inet static bridge_ports none bridge_fd 0 bridge_maxwait 0 address 192.168.100.1/24 up iptables -t nat -C POSTROUTING -s 192.168.100.0/24 ! -o lxcbr1 -j MASQUERADE 2>/dev/null || \ iptables -t nat -A POSTROUTING -s 192.168.100.0/24 ! -o lxcbr1 -j MASQUERADE down iptables -t nat -D POSTROUTING -s 192.168.100.0/24 ! -o lxcbr1 -j MASQUERADE 2>/dev/null || true
ensp1s0 by the default interface on the VM. Replace
192.168.100.0/24 by whatever IP address and subnet you want to use for the
Now bring up the bridge:
Make sure you update the
hosts file to match whichever IP address and subnet
/etc/lxc/default.conf and make sure it looks like the following:
lxc.net.0.type = veth lxc.net.0.link = lxcbr1 lxc.net.0.flags = up lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx
(The hwaddr can be different.) This will ensure that new containers have this configuration by default.
Creating the LXC containers
Start a root shell with
Install the lxc-utils package if you have not done so already:
apt update && apt install lxc-utils
For the time being, it is necessary to manually create each container and to install python3 in it before running the corresponding playbooks. For example, to setup the DNS container, run the following as root:
lxc-create -t download -n dns -- -d debian -r buster -a amd64 lxc-start -n dns chroot /var/lib/lxc/dns/rootfs echo 'nameserver 220.127.116.11' > /etc/resolv.conf apt update apt install -y python3 exit
We are using chroot because the network interfaces have not been setup in the container yet (they will be setup via a playbook).
The containers should be privileged since the CSC currently uses privileged LXC containers. If we switch to unprivileged containers in the future, this repo should be correspondingly updated.
It is also necessary to have Ansible and the Python LXC driver installed on the host where the LXC containers are running. e.g. for Debian:
apt install -y ansible python3-lxc
Now we are ready to run the playbook:
If you see a whole bunch of errors like
RuntimeError: cannot release un-acquired lock
it is safe to ignore those. Here is the GitHub issue if you are interested.