prep switch to ansible

autoautoinstall
Andrew Wang 2 years ago
parent 9f67ff4880
commit 45742ccac7
  1. 28
      README.md
  2. 0
      ansible.cfg
  3. 0
      hosts
  4. 0
      kvm-vm.yml
  5. 0
      playbook.yml
  6. 0
      qemu-vm.yml
  7. 0
      roles/rsync/tasks/main.yml
  8. 0
      roles/rsync/templates/rsyncd-filter.conf.j2
  9. 0
      roles/rsync/templates/rsyncd.conf.j2
  10. 0
      roles/setup/tasks/main.yml
  11. 0
      roles/zfs/tasks/main.yml
  12. 0
      tasks/a
  13. 0
      todo/README.md
  14. 0
      todo/ftp/README.md
  15. 0
      todo/ftp/include/favicon.ico
  16. 0
      todo/ftp/include/header.png
  17. 0
      todo/ftp/include/motd.msg
  18. 0
      todo/ftp/proftpd.conf
  19. 0
      todo/index/README.md
  20. 0
      todo/index/csc-mirror
  21. 0
      todo/index/mirror-index/config.yaml
  22. 0
      todo/index/mirror-index/index.css
  23. 0
      todo/index/mirror-index/index.mako
  24. 0
      todo/index/mirror-index/make-index.py
  25. 0
      todo/nginx/README.md
  26. 0
      todo/nginx/includes/ceph.conf
  27. 0
      todo/nginx/includes/csc-debian.conf
  28. 0
      todo/nginx/includes/mirror.conf
  29. 0
      todo/nginx/includes/ssl.conf
  30. 0
      todo/nginx/includes/ubuntu.conf
  31. 0
      todo/nginx/mime.types
  32. 0
      todo/nginx/nginx.conf
  33. 0
      todo/nginx/sites-available/debian.conf
  34. 0
      todo/nginx/sites-available/default
  35. 0
      todo/nginx/sites-available/default.dpkg-dist
  36. 0
      todo/nginx/sites-available/mirror.conf
  37. 67
      todo/vm-install-op2/main.yml
  38. 95
      todo/vm-install-op2/mirror.txt
  39. 14
      todo/vm-install-op2/mirrornetwork.xml
  40. 30
      todo/vm-install-op2/thing

@ -1,7 +1,7 @@
# Mirror Env
This script helps quickly set up a vm development environment for cs club's mirrors.
## Installation
## VM Installation Option 1 (qmeu-vm.yml)
Install the following dependencies
- qemu
- genisoimage
@ -36,6 +36,27 @@ The default login user has
username: ubuntu
password: ubuntu
```
### Run Playbook
```
ansible-playbook qemu-vm.yml
```
## VM Install Option 2 (kvm)
### Install Packages (debian)
```
$ apt install qemu-kvm libvirt virt-install virt-viewer ansible
```
### Install Packages (archlinux)
```
$ pacman -S qemu libvirt virt-install virt-viewer ansible
```
### Run Playbook
```
ansible-playbook kvm-vm.yml
```
## Post-Installation
The ubunutu autoinstall can only handle basic installation. We require a more powerful tool to configure the post-install environment. For this reason, we will be using ansible.
@ -53,10 +74,8 @@ $ ansible-playbook -K playbook.yml
```
## System Details
For those that are interested.
relevant lsblk output:
```
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda 252:0 0 10G 0 disk
├─vda1 252:1 0 500M 0 part /boot/efi
@ -81,5 +100,4 @@ vdf 252:80 0 10G 0 disk
├─vdf1 252:81 0 10G 0 part
└─vdf9 252:89 0 8M 0 part
```
Drives vda and vdb are for the main filesystem, they use raid1. Drives vdc, vdd, vde and vdf are in a raidz2 zpool.

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.1 KiB

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 14 KiB

@ -0,0 +1,67 @@
---
- name: create mirror vm
hosts: localhost
connection: local
start and set autostart libvirtd
$ systemctl enable --now libvirtd
$ virsh
troubleshooting: try to load the kvm module with `modprobe kvm_intel`
create storage pool for mirror
> pool-define-as mirror dir --target="~/.local/mirror/images"
> pool-build mirror
> pool-start mirror
> pool-autostart mirror
create virtual disks
> vol-create-as mirror mirror_root1.qcow2 10G
> vol-create-as mirror mirror_root2.qcow2 10G
> vol-create-as mirror mirror_disk1.qcow2 5G
> vol-create-as mirror mirror_disk2.qcow2 5G
> vol-create-as mirror mirror_disk3.qcow2 5G
> vol-create-as mirror mirror_disk4.qcow2 5G
(exit virsh shell)
get ubuntu iso
$ curl -O https://releases.ubuntu.com/20.04/ubuntu-20.04.2-live-server-amd64.iso
create the vm
$ virt-install \
--name=mirror \
--memory=2048 \
--vcpus=1 \
--boot uefi \
--location="./ubuntu-20.04.2-live-server-amd64.iso" \
--disk vol=mirror/mirror_root1.qcow2 \
--disk vol=mirror/mirror_root2.qcow2 \
--disk vol=mirror/mirror_disk1.qcow2 \
--disk vol=mirror/mirror_disk2.qcow2 \
--disk vol=mirror/mirror_disk3.qcow2 \
--disk vol=mirror/mirror_disk4.qcow2 \
interface should automatically come up but can also use
$ virt-viewer --domain-name mirror
if vm is on a remote machine
$ virt-viewer --connect qemu+ssh://user@host.example.com/system vmnamehere
for now just have a folder of screenshots
- change name of lvm volume from "lv0-root" to just "root"
troubleshooting: ignore "failed to unmount /cdrom" and just ctrl+c in viewer
login into your created user (from install), change to root, and create password for root
$ sudo su
type in your password
$ passwd
create root password
create zpool (use /dev/disk/by-id/.. instead)
$ apt update
$ apt install zfsutils-linux nginx rsync
$ mkdir -p /mirror/root/.cscmirror
$ zpool create -f -m /mirror/root/.cscmirror cscmirror raidz2 /dev/vdc /dev/vdd /dev/vde /dev/vdf
may already be mounted but just to be sure
$ zfs mount -a

@ -0,0 +1,95 @@
remarks
- vm will not have autostart
- storage req: 40GB
- root size: 10GB (x2 for raid 1)
- storage size: 5GB (x4 for raid-z2)
install (debian)
$ apt install qemu-kvm libvirt virt-install virt-viewer
install (arch)
$ pacman -S qemu libvirt virt-install virt-viewer
start and set autostart libvirtd
$ systemctl enable --now libvirtd
$ virsh
troubleshooting: try to load the kvm module with `modprobe kvm_intel`
create storage pool for mirror
> pool-define-as mirror dir --target="~/.local/mirror/images"
> pool-build mirror
> pool-start mirror
> pool-autostart mirror
create virtual disks
> vol-create-as mirror mirror_root1.qcow2 10G
> vol-create-as mirror mirror_root2.qcow2 10G
> vol-create-as mirror mirror_disk1.qcow2 5G
> vol-create-as mirror mirror_disk2.qcow2 5G
> vol-create-as mirror mirror_disk3.qcow2 5G
> vol-create-as mirror mirror_disk4.qcow2 5G
(exit virsh shell)
get ubuntu iso
$ curl -O https://releases.ubuntu.com/20.04/ubuntu-20.04.2-live-server-amd64.iso
create the vm
$ virt-install \
--name=mirror \
--memory=2048 \
--vcpus=1 \
--boot uefi \
--location="./ubuntu-20.04.2-live-server-amd64.iso" \
--disk vol=mirror/mirror_root1.qcow2 \
--disk vol=mirror/mirror_root2.qcow2 \
--disk vol=mirror/mirror_disk1.qcow2 \
--disk vol=mirror/mirror_disk2.qcow2 \
--disk vol=mirror/mirror_disk3.qcow2 \
--disk vol=mirror/mirror_disk4.qcow2 \
interface should automatically come up but can also use
$ virt-viewer --domain-name mirror
if vm is on a remote machine
$ virt-viewer --connect qemu+ssh://user@host.example.com/system vmnamehere
for now just have a folder of screenshots
- change name of lvm volume from "lv0-root" to just "root"
troubleshooting: ignore "failed to unmount /cdrom" and just ctrl+c in viewer
login into your created user (from install), change to root, and create password for root
$ sudo su
type in your password
$ passwd
create root password
create zpool (use /dev/disk/by-id/.. instead)
$ apt update
$ apt install zfsutils-linux nginx rsync
$ mkdir -p /mirror/root/.cscmirror
$ zpool create -f -m /mirror/root/.cscmirror cscmirror raidz2 /dev/vdc /dev/vdd /dev/vde /dev/vdf
may already be mounted but just to be sure
$ zfs mount -a
TODO
- configure symlinking from /mirror/root/.cscmirror to /mirror/root
- configure ssh for push sync
- configure nginx
- configure rsync
- set motd?
- configure FTP (using proftpd)
- Merlin in ~mirror/merlin -> /home/mirror/merlin
- ~mirror/merlin/arthur.py status
- ~mirror/merlin/arthur.py sync:PROJECT_NAME
- Sync Scripts in ~mirror/bin (and repo, but repo has less)
- take params local_dir rsync_host rsync_dir
- check out csclub wiki mirror administration for adding a new project and project sync
- ask if can ignore /mirror/newroot
- mirror hosts
- http://mirror.csclub.uwaterloo.ca (the mirror)
- http://ca.releases.ubuntu.com (ubuntu releases)
- http://ca.ceph.com (ceph releases)
- http://debian.csclub.uwaterloo.ca (csclub's debian packages)

@ -0,0 +1,14 @@
<network>
<name>mirbr0</name>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='mirbr0' stp='on' delay='0'/>
<ip address='192.168.123.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.123.2' end='192.168.123.254'/>
</dhcp>
</ip>
</network>

@ -0,0 +1,30 @@
create network and storage interface?
<interface type='network'>
<mac address='52:54:00:64:dd:f2'/>
<source network='default' bridge='virbr0'/>
<target dev='vnet0'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>
<graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
<image compression='off'/>
</graphics>
<graphics type='vnc' port='5901' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
</graphics>
<graphics type='spice' port='5902' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
<image compression='off'/>
</graphics>
<graphics type='vnc' port='5903' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
</graphics>
Loading…
Cancel
Save