commit bfa78ef0bea18f05643d5db6eb950c54751193ab Author: Max Erenberg <> Date: Mon Nov 29 09:17:07 2021 -0500 add docs for CloudStack and vhosts diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..7f24b8d --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +site/ +/venv/ +*.swp diff --git a/README.md b/README.md new file mode 100644 index 0000000..0ed24ba --- /dev/null +++ b/README.md @@ -0,0 +1,20 @@ +# docs +This is the repository for [https://docs.cloud.csclub.uwaterloo.ca](https://docs.cloud.csclub.uwaterloo.ca). + +## Contributing +This is a [MkDocs](https://www.mkdocs.org) site, so all pages are writen in Markdown. + +Install the prerequisites: +```sh +pip install mkdocs mkdocs-material pymdown-extensions +``` + +Start a local web server (http://localhost:8000): +```sh +mkdocs serve +``` + +Building the site: +``` +mkdocs build +``` diff --git a/docs/cloudstack.md b/docs/cloudstack.md new file mode 100644 index 0000000..3cd8f1f --- /dev/null +++ b/docs/cloudstack.md @@ -0,0 +1,167 @@ +# CloudStack + +[Apache CloudStack](https://cloudstack.apache.org/) is an open-source +cloud computing platform which we use to allow members to create their +own virtual machines (VMs). + +## Activating your cloud account +Before using CloudStack, you must activate your cloud account. +Log into any [general-use machine](https://wiki.csclub.uwaterloo.ca/Machine_List) +and run the following: +```sh +ceo cloud account activate +``` +Now visit [https://cloud.csclub.uwaterloo.ca](https://cloud.csclub.uwaterloo.ca) +and login with your CSC username and password. For the domain, just enter 'Members' +(no quotes). + +CloudStack login + +## Account resource limits +As of this writing, the CloudStack resource limits for each member are: + +* 8 CPU cores +* 8 GB of RAM +* 40 GB of disk space + +If you wish to acquire more resources, please send an email to the Systems Committee +with a brief justification. + +## Adding your SSH key +You will want to do this *before* creating a VM. + +!!! note + + The rest of this section assumes that you already have an SSH key pair. + If you do not have one yet, please create one first; there are plenty + of good tutorials online on how to do this + ([here](https://www.howtogeek.com/762863/how-to-generate-ssh-keys-in-windows-10-and-windows-11/) + is one example). + +Once you've logged into CloudStack, click on the 'Compute' button on the left-hand +panel (depending on how wide your screen is, you may only see a cloud icon), +then click 'SSH key pairs'. Click the 'Create a SSH Key Pair' button. + +CloudStack SSH key pair button + +Now you will need to add your public key (this is the contents of the file which ends +with '.pub'). + +You can name the key pair anything you like; just make sure you remember the name. + +CloudStack add SSH key pair + +Click OK once you have pasted your public key into the text box. + +## Creating a VM +Click on the 'Compute' button on the left-hand panel, then click 'Instances'. Click +the 'Add Instance' button. + +CloudStack Add Instance button + +Under the 'Template/ISO' section, choose the OS which you want your VM to run. + +!!! tip + + Don't see your favourite OS listed? No problem! Just send an email to the + [Systems Committee](mailto:syscom@csclub.uwaterloo.ca) requesting your OS + to be added. We require that the OS must already have a publically available + cloud image which has been prepared with [cloud-init](https://cloud-init.io/). + +**Important**: Make sure to toggle the 'Override Root Disk Size' option and set it +to something reasonable (see [Account resource limits](#account-resource-limits) for +the maximum disk space which you can use.) The default root disk size for cloud +images is usually very small (e.g. 2GB), so you will definitely want to change it. + +CloudStack Root Disk Size + +Now you need to choose a Compute Offering. If you are sure not sure how much +you need, we recommend that you start off with a Small or Medium instance; you +can always upgrade later if necessary. + + +CloudStack Compute Offering + +You probably do not need to add an external disk. Be aware that if you do add one, +this counts towards your total disk quota. + + +CloudStack Data Disk + +Make sure that you do **NOT** place the VM in the default security group, because +this blocks all ingress traffic by default. + +CloudStack Security Groups + +Choose the SSH keypair which you created earlier: + +CloudStack Choose SSH keypair + +Enter a name for your VM: + +CloudStack VM details + +Now press 'Launch Virtual Machine'. + +If all goes well, you should see your VM running from the 'Instances' dashboard: +CloudStack running instances + +## Accessing your VM +All VMs will obtain an IP address from the 172.19.134.0/24 subnet (MC VLAN 425). +As this is a private IP range, you can only access this *directly* from on campus. If you +are not on campus, you can still access your VM via the following methods: + +* use the [Campus VPN](https://uwaterloo.ca/information-systems-technology/services/virtual-private-network-vpn) +* use a general-use CSC machine as a jump host + +The second option is generally more convenient. The idea is to SSH into a CSC +general-use machine first, then SSH from there into your VM. + +On any CSC general-use machine, copy your SSH key to the `.ssh` folder in your +home directory (e.g. via scp). Make sure that it is only readable by you (e.g. +run `chmod 600` on it). Running `ls -l ~/.ssh` should show something like this: + + total 16 + -rw-r--r-- 1 ctdalek ctdalek 918 Oct 14 12:05 authorized_keys + -rw-r--r-- 1 ctdalek ctdalek 25 Oct 27 14:03 config + -rw------- 1 ctdalek ctdalek 1896 Sep 3 18:23 id_rsa + -rw-r----- 1 ctdalek ctdalek 415 Sep 3 18:23 id_rsa.pub + -rw-r--r-- 1 ctdalek ctdalek 459 Nov 24 12:29 known_hosts + +Note how the `id_rsa` file is not world-readable. + +Now, from the CSC machine, you can SSH into your VM by running a command like the following: +```sh +ssh debian@172.19.134.121 +``` +Replace `172.19.13.121` with the IP address of your VM, and replace `debian` with the +default username of the OS which you chose: + +* Debian: `debian` +* Ubuntu: `ubuntu` +* CentOS: `centos` + +Once you have logged in, you can run `sudo -s` to become the root user. + +See [SSH tricks](../ssh-tricks) for some useful SSH tricks. + +## Next steps +Congratulations, you've created your VM! ...So what do you do now? + +That's completely up to you! Since you can become the root user inside the +VM, you can install and run any software you want (just keep in mind that +you must still follow the +[Machine Usage Agreement](https://csclub.uwaterloo.ca/resources/machine-usage-agreement/)). + +Here are some ideas to get you started: + +* Install [Docker](https://docs.docker.com/engine/install/) and run some containers +* Install [Apache](https://httpd.apache.org/) or [NGINX](http://nginx.org/) and + serve a static website (e.g. a blog) +* Run a Node.js/Python/Golang/whatever web application +* Install [k0s](https://k0sproject.io/) and run a small Kubernetes cluster + +The world's your oyster! 😊 + +If you plan on hosting a publicly available website, you will want to create +a virtual host. See [Virtual Hosting](../vhosts) for details. diff --git a/docs/img/cloudstack_add_instance_button.png b/docs/img/cloudstack_add_instance_button.png new file mode 100644 index 0000000..2ca224c Binary files /dev/null and b/docs/img/cloudstack_add_instance_button.png differ diff --git a/docs/img/cloudstack_add_ssh_key_pair.png b/docs/img/cloudstack_add_ssh_key_pair.png new file mode 100644 index 0000000..e024bbe Binary files /dev/null and b/docs/img/cloudstack_add_ssh_key_pair.png differ diff --git a/docs/img/cloudstack_choose_ssh_keypair.png b/docs/img/cloudstack_choose_ssh_keypair.png new file mode 100644 index 0000000..65ac149 Binary files /dev/null and b/docs/img/cloudstack_choose_ssh_keypair.png differ diff --git a/docs/img/cloudstack_compute_offering.png b/docs/img/cloudstack_compute_offering.png new file mode 100644 index 0000000..0af3ad3 Binary files /dev/null and b/docs/img/cloudstack_compute_offering.png differ diff --git a/docs/img/cloudstack_data_disk.png b/docs/img/cloudstack_data_disk.png new file mode 100644 index 0000000..7f295f5 Binary files /dev/null and b/docs/img/cloudstack_data_disk.png differ diff --git a/docs/img/cloudstack_login.png b/docs/img/cloudstack_login.png new file mode 100644 index 0000000..399a232 Binary files /dev/null and b/docs/img/cloudstack_login.png differ diff --git a/docs/img/cloudstack_root_disk_size.png b/docs/img/cloudstack_root_disk_size.png new file mode 100644 index 0000000..4bda3b3 Binary files /dev/null and b/docs/img/cloudstack_root_disk_size.png differ diff --git a/docs/img/cloudstack_running_instances.png b/docs/img/cloudstack_running_instances.png new file mode 100644 index 0000000..843348c Binary files /dev/null and b/docs/img/cloudstack_running_instances.png differ diff --git a/docs/img/cloudstack_security_groups.png b/docs/img/cloudstack_security_groups.png new file mode 100644 index 0000000..7f9c9d9 Binary files /dev/null and b/docs/img/cloudstack_security_groups.png differ diff --git a/docs/img/cloudstack_ssh_key_pair_button.png b/docs/img/cloudstack_ssh_key_pair_button.png new file mode 100644 index 0000000..53ce5da Binary files /dev/null and b/docs/img/cloudstack_ssh_key_pair_button.png differ diff --git a/docs/img/cloudstack_vm_details.png b/docs/img/cloudstack_vm_details.png new file mode 100644 index 0000000..989ce80 Binary files /dev/null and b/docs/img/cloudstack_vm_details.png differ diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..bd3a120 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,28 @@ +# Welcome to the CSC Cloud! + +The CSC cloud is a collection of compute, network and storage services +available to CSC members. + +## Available services + +* [CloudStack](cloudstack) - Virtual machines on demand. +* [Virtual hosting](vhosts) - Virtual domain name hosting for websites. +* Kubernetes (coming soon!) + +## Getting Started + +First, make sure that [you are a CSC member](https://csclub.uwaterloo.ca/get-involved/) +and that your membership has not expired. + +Here is a list of general-use CSC machines which you may SSH into after becoming +a member: +[https://wiki.csclub.uwaterloo.ca/Machine_List](https://wiki.csclub.uwaterloo.ca/Machine_List) + +If you wish to see the terms for which your membership is valid, you can run the +following command from any general-use machine: +```sh +ceo members get +``` + +To start creating your own virtual machines, please see the [CloudStack](cloudstack) +page for more details. diff --git a/docs/ssh-tricks.md b/docs/ssh-tricks.md new file mode 100644 index 0000000..1f58ce5 --- /dev/null +++ b/docs/ssh-tricks.md @@ -0,0 +1,99 @@ +# SSH Tricks + +Here are some useful SSH tricks for accessing your VM. + +The commands below assume that the SSH public key on your personal machine +has already been copied to two places: + +1. the `~/.ssh/authorized_keys` file in your CSC home directory +2. the `~/.ssh/authorized_keys` file of the default user in your VM + +!!! Note + If you are having trouble with any of the commands below, please don't + hesitate to ask the [Systems Committee](mailto:syscom@csclub.uwaterloo.ca) + for assistance. + +Most of the "tricks" below require ProxyJump to have been setup, so we suggest +reading that first. + +## ProxyJump +To avoid having to manually SSH to a CSC machine before SSH'ing to your +VM, you can use the ProxyJump directive. For example, let's say your VM's +IP address is `172.19.134.121`, and you want to use `corn-syrup` as a jump +host. + +Add a snippet similar to the following in your `~/.ssh/config` (on your +personal machine): +```sh +Host corn-syrup + HostName corn-syrup.csclub.uwaterloo.ca + # Replace this with your username + User ctdalek +Host ctdalek-vm1 + # Replace this with the IP address of your VM + HostName 172.19.134.121 + ProxyJump corn-syrup + # Replace this with the default user in your VM + User debian +``` + +Now you can connect to your VM by running +```sh +ssh ctdalek-vm1 +``` + +!!! Note + If the name of your SSH key is not one of the default names (e.g. id_rsa, + id_ed25519), you may also need to specify the `IdentityFile` option. + +## Port forwarding +Let's say you have a process bound to `localhost:8000` in your VM, and you'd like +to access it from your personal machine. Then you just need to run the following: +```sh +ssh -L 8000:localhost:8000 ctdalek-vm1 +``` +This will forward requests to `localhost:8000` on your personal machine to +`localhost:8000` on your VM. + +If you want to fork the process to the background, here's one way to do it: +```sh +ssh -L 8000:localhost:8000 -CNfq ctdalek-vm1 +``` +Explanation: + +* `-C`: compress (saves bandwidth) +* `-N`: don't execute a command on the server +* `-f`: fork the SSH process to the background +* `-q`: quiet (silences output) + +## Reverse port forwarding +Let's say you have a process bound to `localhost:8000` on your personal machine, +and you'd like to access it from your VM. Instead of using `-L`, you want to use +`-R` instead: +```sh +ssh -R 8000:localhost:8000 -CNfq ctdalek-vm1 +``` +This will forward requests to `localhost:8000` in the VM to `localhost:8000` on +your personal machine. + +## SOCKS proxy +You probably won't need this one, but it's good to know. This basically allows +you to use a CSC machine as a proxy for *all* of your Internet traffic for a +particular application. It's useful when you need to access a website which +is only available from the campus network. + +First, let's run a SOCKS proxy on e.g. `localhost:8132`: +```sh +ssh -D 8132 -CNfq corn-syrup +``` +You now need to configure your application to use the proxy. For example, in +Firefox, you can do the following: + +* Visit `about:preferences` in the URL bar +* Scroll to the bottom, and click the Settings button under 'Network Settings' +* Select 'Manual proxy configuration' +* Enter 'localhost' as the SOCKS Host, and 8132 for the port. Also make sure + 'SOCKS v5' is selected. + +After pressing 'OK', you should now be able to visit websites using a campus +IP address. diff --git a/docs/vhosts.md b/docs/vhosts.md new file mode 100644 index 0000000..bd306a8 --- /dev/null +++ b/docs/vhosts.md @@ -0,0 +1,65 @@ +# Virtual Hosting + +[Virtual hosting](https://en.wikipedia.org/wiki/Virtual_hosting) is a method +for hosting multiple websites with different domain names on a single web +server. The idea is to redirect external traffic from the CSC web server to +your VM based on the domain name. If you are hosting a website on your VM +and you would like it to be publically accessible, you will need to create +a vhost record for your VM. + +## Domain limitations +Each member may create up to 10 vhosts whose domains end with +`.csclub.cloud`. + +For example, if your username is `ctdalek`, then the following vhost domains +are all valid: + +* ctdalek.csclub.cloud +* www.ctdalek.csclub.cloud +* abc.def.ctdalek.csclub.cloud + +## Creating a vhost record +Let's say your VM's IP address is 172.19.134.121 and you wish to use the domain +`ctdalek.csclub.cloud`. You must already have an application running on port 80 +in your VM (e.g. NGINX). + +Login to a [CSC general-use machine](https://wiki.csclub.uwaterloo.ca/Machine_List) +and run the following: +```sh +ceo cloud vhosts add ctdalek.csclub.cloud 172.19.134.121 +``` +In addition to creating the vhost record, this command will also provision a TLS +certificate for your website (from either [ZeroSSL](https://zerossl.com/) or +[Let's Encrypt](https://letsencrypt.org/)). You should now be able to visit your +website over HTTPS from your browser, e.g. `https://ctdalek.csclub.cloud`. + +!!! warning + To avoid having to reload our web server too frequently, and to reduce the risk + of getting banned by our ACME service provider, we have set a rate limit on the + command above for **once every five minutes**. We apologize for the inconvnience. + +To view your vhost records, run the following: +```sh +ceo cloud vhosts list +``` + +To delete a vhost record, run the following: +```sh +ceo cloud vhosts delete ctdalek.csclub.cloud +``` + +!!! info + Multiple domain names can point to the same IP address, but a single domain name + can only point to one IP address. + +## Using a custom domain name +If you wish to use a custom domain name which you have purchased from an external +registrar, please make it point to the IP addresses of `csclub.cloud`. As of this writing, +this is: + +* IPv4: `129.97.134.10` +* IPv6: `2620:101:f000:4901:c5c::10ad` + +Then, please send an email to the [Systems Committee](mailto:syscom@csclub.uwaterloo.ca) +requesting a vhost record for your domain. Make sure to also provide the IP address +of your VM. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..9e1a074 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,34 @@ +site_name: CSC Cloud Docs +site_url: https://docs.cloud.csclub.uwaterloo.ca/ +nav: + - Home: index.md + - CloudStack: cloudstack.md + - Virtual Hosting: vhosts.md + - SSH Tricks: ssh-tricks.md + +# pip install pymdown-extensions +markdown_extensions: + - admonition + - pymdownx.details + - pymdownx.superfences + - pymdownx.highlight + - pymdownx.inlinehilite + +# pip install mkdocs-material +theme: + name: material + favicon: https://git.csclub.uwaterloo.ca/public/csc-propaganda/raw/branch/master/csc-logos/csc-logo.svg + palette: + - scheme: default + toggle: + icon: material/toggle-switch-off-outline + name: Switch to dark mode + - scheme: slate + toggle: + icon: material/toggle-switch + name: Switch to light mode + +repo_url: https://git.csclub.uwaterloo.ca/cloud/docs +repo_name: cloud/docs + +copyright: Copyright © 2021 Computer Science Club of the University of Waterloo