Browse Source

add docs for CloudStack and vhosts

master
Max Erenberg 9 months ago
commit
bfa78ef0be
  1. 3
      .gitignore
  2. 20
      README.md
  3. 167
      docs/cloudstack.md
  4. BIN
      docs/img/cloudstack_add_instance_button.png
  5. BIN
      docs/img/cloudstack_add_ssh_key_pair.png
  6. BIN
      docs/img/cloudstack_choose_ssh_keypair.png
  7. BIN
      docs/img/cloudstack_compute_offering.png
  8. BIN
      docs/img/cloudstack_data_disk.png
  9. BIN
      docs/img/cloudstack_login.png
  10. BIN
      docs/img/cloudstack_root_disk_size.png
  11. BIN
      docs/img/cloudstack_running_instances.png
  12. BIN
      docs/img/cloudstack_security_groups.png
  13. BIN
      docs/img/cloudstack_ssh_key_pair_button.png
  14. BIN
      docs/img/cloudstack_vm_details.png
  15. 28
      docs/index.md
  16. 99
      docs/ssh-tricks.md
  17. 65
      docs/vhosts.md
  18. 34
      mkdocs.yml

3
.gitignore vendored

@ -0,0 +1,3 @@
site/
/venv/
*.swp

20
README.md

@ -0,0 +1,20 @@
# docs
This is the repository for [https://docs.cloud.csclub.uwaterloo.ca](https://docs.cloud.csclub.uwaterloo.ca).
## Contributing
This is a [MkDocs](https://www.mkdocs.org) site, so all pages are writen in Markdown.
Install the prerequisites:
```sh
pip install mkdocs mkdocs-material pymdown-extensions
```
Start a local web server (http://localhost:8000):
```sh
mkdocs serve
```
Building the site:
```
mkdocs build
```

167
docs/cloudstack.md

@ -0,0 +1,167 @@
# CloudStack
[Apache CloudStack](https://cloudstack.apache.org/) is an open-source
cloud computing platform which we use to allow members to create their
own virtual machines (VMs).
## Activating your cloud account
Before using CloudStack, you must activate your cloud account.
Log into any [general-use machine](https://wiki.csclub.uwaterloo.ca/Machine_List)
and run the following:
```sh
ceo cloud account activate
```
Now visit [https://cloud.csclub.uwaterloo.ca](https://cloud.csclub.uwaterloo.ca)
and login with your CSC username and password. For the domain, just enter 'Members'
(no quotes).
<img src="../img/cloudstack_login.png" alt="CloudStack login" width="400" />
## Account resource limits
As of this writing, the CloudStack resource limits for each member are:
* 8 CPU cores
* 8 GB of RAM
* 40 GB of disk space
If you wish to acquire more resources, please send an email to the Systems Committee
with a brief justification.
## Adding your SSH key
You will want to do this *before* creating a VM.
!!! note
The rest of this section assumes that you already have an SSH key pair.
If you do not have one yet, please create one first; there are plenty
of good tutorials online on how to do this
([here](https://www.howtogeek.com/762863/how-to-generate-ssh-keys-in-windows-10-and-windows-11/)
is one example).
Once you've logged into CloudStack, click on the 'Compute' button on the left-hand
panel (depending on how wide your screen is, you may only see a cloud icon),
then click 'SSH key pairs'. Click the 'Create a SSH Key Pair' button.
<img src="../img/cloudstack_ssh_key_pair_button.png" alt="CloudStack SSH key pair button" width="800" />
Now you will need to add your public key (this is the contents of the file which ends
with '.pub').
You can name the key pair anything you like; just make sure you remember the name.
<img src="../img/cloudstack_add_ssh_key_pair.png" alt="CloudStack add SSH key pair" width="400" />
Click OK once you have pasted your public key into the text box.
## Creating a VM
Click on the 'Compute' button on the left-hand panel, then click 'Instances'. Click
the 'Add Instance' button.
<img src="../img/cloudstack_add_instance_button.png" alt="CloudStack Add Instance button" width="800" />
Under the 'Template/ISO' section, choose the OS which you want your VM to run.
!!! tip
Don't see your favourite OS listed? No problem! Just send an email to the
[Systems Committee](mailto:syscom@csclub.uwaterloo.ca) requesting your OS
to be added. We require that the OS must already have a publically available
cloud image which has been prepared with [cloud-init](https://cloud-init.io/).
**Important**: Make sure to toggle the 'Override Root Disk Size' option and set it
to something reasonable (see [Account resource limits](#account-resource-limits) for
the maximum disk space which you can use.) The default root disk size for cloud
images is usually very small (e.g. 2GB), so you will definitely want to change it.
<img src="../img/cloudstack_root_disk_size.png" alt="CloudStack Root Disk Size" width="700" />
Now you need to choose a Compute Offering. If you are sure not sure how much
you need, we recommend that you start off with a Small or Medium instance; you
can always upgrade later if necessary.
<img src="../img/cloudstack_compute_offering.png" alt="CloudStack Compute Offering" width="700" />
You probably do not need to add an external disk. Be aware that if you do add one,
this counts towards your total disk quota.
<img src="../img/cloudstack_data_disk.png" alt="CloudStack Data Disk" width="700" />
Make sure that you do **NOT** place the VM in the default security group, because
this blocks all ingress traffic by default.
<img src="../img/cloudstack_security_groups.png" alt="CloudStack Security Groups" width="700" />
Choose the SSH keypair which you created earlier:
<img src="../img/cloudstack_choose_ssh_keypair.png" alt="CloudStack Choose SSH keypair" width="700" />
Enter a name for your VM:
<img src="../img/cloudstack_vm_details.png" alt="CloudStack VM details" width="700" />
Now press 'Launch Virtual Machine'.
If all goes well, you should see your VM running from the 'Instances' dashboard:
<img src="../img/cloudstack_running_instances.png" alt="CloudStack running instances" width="800" />
## Accessing your VM
All VMs will obtain an IP address from the 172.19.134.0/24 subnet (MC VLAN 425).
As this is a private IP range, you can only access this *directly* from on campus. If you
are not on campus, you can still access your VM via the following methods:
* use the [Campus VPN](https://uwaterloo.ca/information-systems-technology/services/virtual-private-network-vpn)
* use a general-use CSC machine as a jump host
The second option is generally more convenient. The idea is to SSH into a CSC
general-use machine first, then SSH from there into your VM.
On any CSC general-use machine, copy your SSH key to the `.ssh` folder in your
home directory (e.g. via scp). Make sure that it is only readable by you (e.g.
run `chmod 600` on it). Running `ls -l ~/.ssh` should show something like this:
total 16
-rw-r--r-- 1 ctdalek ctdalek 918 Oct 14 12:05 authorized_keys
-rw-r--r-- 1 ctdalek ctdalek 25 Oct 27 14:03 config
-rw------- 1 ctdalek ctdalek 1896 Sep 3 18:23 id_rsa
-rw-r----- 1 ctdalek ctdalek 415 Sep 3 18:23 id_rsa.pub
-rw-r--r-- 1 ctdalek ctdalek 459 Nov 24 12:29 known_hosts
Note how the `id_rsa` file is not world-readable.
Now, from the CSC machine, you can SSH into your VM by running a command like the following:
```sh
ssh debian@172.19.134.121
```
Replace `172.19.13.121` with the IP address of your VM, and replace `debian` with the
default username of the OS which you chose:
* Debian: `debian`
* Ubuntu: `ubuntu`
* CentOS: `centos`
Once you have logged in, you can run `sudo -s` to become the root user.
See [SSH tricks](../ssh-tricks) for some useful SSH tricks.
## Next steps
Congratulations, you've created your VM! ...So what do you do now?
That's completely up to you! Since you can become the root user inside the
VM, you can install and run any software you want (just keep in mind that
you must still follow the
[Machine Usage Agreement](https://csclub.uwaterloo.ca/resources/machine-usage-agreement/)).
Here are some ideas to get you started:
* Install [Docker](https://docs.docker.com/engine/install/) and run some containers
* Install [Apache](https://httpd.apache.org/) or [NGINX](http://nginx.org/) and
serve a static website (e.g. a blog)
* Run a Node.js/Python/Golang/whatever web application
* Install [k0s](https://k0sproject.io/) and run a small Kubernetes cluster
The world's your oyster! 😊
If you plan on hosting a publicly available website, you will want to create
a virtual host. See [Virtual Hosting](../vhosts) for details.

BIN
docs/img/cloudstack_add_instance_button.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

BIN
docs/img/cloudstack_add_ssh_key_pair.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

BIN
docs/img/cloudstack_choose_ssh_keypair.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

BIN
docs/img/cloudstack_compute_offering.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

BIN
docs/img/cloudstack_data_disk.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

BIN
docs/img/cloudstack_login.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

BIN
docs/img/cloudstack_root_disk_size.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

BIN
docs/img/cloudstack_running_instances.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

BIN
docs/img/cloudstack_security_groups.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

BIN
docs/img/cloudstack_ssh_key_pair_button.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

BIN
docs/img/cloudstack_vm_details.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

28
docs/index.md

@ -0,0 +1,28 @@
# Welcome to the CSC Cloud!
The CSC cloud is a collection of compute, network and storage services
available to CSC members.
## Available services
* [CloudStack](cloudstack) - Virtual machines on demand.
* [Virtual hosting](vhosts) - Virtual domain name hosting for websites.
* Kubernetes (coming soon!)
## Getting Started
First, make sure that [you are a CSC member](https://csclub.uwaterloo.ca/get-involved/)
and that your membership has not expired.
Here is a list of general-use CSC machines which you may SSH into after becoming
a member:
[https://wiki.csclub.uwaterloo.ca/Machine_List](https://wiki.csclub.uwaterloo.ca/Machine_List)
If you wish to see the terms for which your membership is valid, you can run the
following command from any general-use machine:
```sh
ceo members get <your_username>
```
To start creating your own virtual machines, please see the [CloudStack](cloudstack)
page for more details.

99
docs/ssh-tricks.md

@ -0,0 +1,99 @@
# SSH Tricks
Here are some useful SSH tricks for accessing your VM.
The commands below assume that the SSH public key on your personal machine
has already been copied to two places:
1. the `~/.ssh/authorized_keys` file in your CSC home directory
2. the `~/.ssh/authorized_keys` file of the default user in your VM
!!! Note
If you are having trouble with any of the commands below, please don't
hesitate to ask the [Systems Committee](mailto:syscom@csclub.uwaterloo.ca)
for assistance.
Most of the "tricks" below require ProxyJump to have been setup, so we suggest
reading that first.
## ProxyJump
To avoid having to manually SSH to a CSC machine before SSH'ing to your
VM, you can use the ProxyJump directive. For example, let's say your VM's
IP address is `172.19.134.121`, and you want to use `corn-syrup` as a jump
host.
Add a snippet similar to the following in your `~/.ssh/config` (on your
personal machine):
```sh
Host corn-syrup
HostName corn-syrup.csclub.uwaterloo.ca
# Replace this with your username
User ctdalek
Host ctdalek-vm1
# Replace this with the IP address of your VM
HostName 172.19.134.121
ProxyJump corn-syrup
# Replace this with the default user in your VM
User debian
```
Now you can connect to your VM by running
```sh
ssh ctdalek-vm1
```
!!! Note
If the name of your SSH key is not one of the default names (e.g. id_rsa,
id_ed25519), you may also need to specify the `IdentityFile` option.
## Port forwarding
Let's say you have a process bound to `localhost:8000` in your VM, and you'd like
to access it from your personal machine. Then you just need to run the following:
```sh
ssh -L 8000:localhost:8000 ctdalek-vm1
```
This will forward requests to `localhost:8000` on your personal machine to
`localhost:8000` on your VM.
If you want to fork the process to the background, here's one way to do it:
```sh
ssh -L 8000:localhost:8000 -CNfq ctdalek-vm1
```
Explanation:
* `-C`: compress (saves bandwidth)
* `-N`: don't execute a command on the server
* `-f`: fork the SSH process to the background
* `-q`: quiet (silences output)
## Reverse port forwarding
Let's say you have a process bound to `localhost:8000` on your personal machine,
and you'd like to access it from your VM. Instead of using `-L`, you want to use
`-R` instead:
```sh
ssh -R 8000:localhost:8000 -CNfq ctdalek-vm1
```
This will forward requests to `localhost:8000` in the VM to `localhost:8000` on
your personal machine.
## SOCKS proxy
You probably won't need this one, but it's good to know. This basically allows
you to use a CSC machine as a proxy for *all* of your Internet traffic for a
particular application. It's useful when you need to access a website which
is only available from the campus network.
First, let's run a SOCKS proxy on e.g. `localhost:8132`:
```sh
ssh -D 8132 -CNfq corn-syrup
```
You now need to configure your application to use the proxy. For example, in
Firefox, you can do the following:
* Visit `about:preferences` in the URL bar
* Scroll to the bottom, and click the Settings button under 'Network Settings'
* Select 'Manual proxy configuration'
* Enter 'localhost' as the SOCKS Host, and 8132 for the port. Also make sure
'SOCKS v5' is selected.
After pressing 'OK', you should now be able to visit websites using a campus
IP address.

65
docs/vhosts.md

@ -0,0 +1,65 @@
# Virtual Hosting
[Virtual hosting](https://en.wikipedia.org/wiki/Virtual_hosting) is a method
for hosting multiple websites with different domain names on a single web
server. The idea is to redirect external traffic from the CSC web server to
your VM based on the domain name. If you are hosting a website on your VM
and you would like it to be publically accessible, you will need to create
a vhost record for your VM.
## Domain limitations
Each member may create up to 10 vhosts whose domains end with
`<username>.csclub.cloud`.
For example, if your username is `ctdalek`, then the following vhost domains
are all valid:
* ctdalek.csclub.cloud
* www.ctdalek.csclub.cloud
* abc.def.ctdalek.csclub.cloud
## Creating a vhost record
Let's say your VM's IP address is 172.19.134.121 and you wish to use the domain
`ctdalek.csclub.cloud`. You must already have an application running on port 80
in your VM (e.g. NGINX).
Login to a [CSC general-use machine](https://wiki.csclub.uwaterloo.ca/Machine_List)
and run the following:
```sh
ceo cloud vhosts add ctdalek.csclub.cloud 172.19.134.121
```
In addition to creating the vhost record, this command will also provision a TLS
certificate for your website (from either [ZeroSSL](https://zerossl.com/) or
[Let's Encrypt](https://letsencrypt.org/)). You should now be able to visit your
website over HTTPS from your browser, e.g. `https://ctdalek.csclub.cloud`.
!!! warning
To avoid having to reload our web server too frequently, and to reduce the risk
of getting banned by our ACME service provider, we have set a rate limit on the
command above for **once every five minutes**. We apologize for the inconvnience.
To view your vhost records, run the following:
```sh
ceo cloud vhosts list
```
To delete a vhost record, run the following:
```sh
ceo cloud vhosts delete ctdalek.csclub.cloud
```
!!! info
Multiple domain names can point to the same IP address, but a single domain name
can only point to one IP address.
## Using a custom domain name
If you wish to use a custom domain name which you have purchased from an external
registrar, please make it point to the IP addresses of `csclub.cloud`. As of this writing,
this is:
* IPv4: `129.97.134.10`
* IPv6: `2620:101:f000:4901:c5c::10ad`
Then, please send an email to the [Systems Committee](mailto:syscom@csclub.uwaterloo.ca)
requesting a vhost record for your domain. Make sure to also provide the IP address
of your VM.

34
mkdocs.yml

@ -0,0 +1,34 @@
site_name: CSC Cloud Docs
site_url: https://docs.cloud.csclub.uwaterloo.ca/
nav:
- Home: index.md
- CloudStack: cloudstack.md
- Virtual Hosting: vhosts.md
- SSH Tricks: ssh-tricks.md
# pip install pymdown-extensions
markdown_extensions:
- admonition
- pymdownx.details
- pymdownx.superfences
- pymdownx.highlight
- pymdownx.inlinehilite
# pip install mkdocs-material
theme:
name: material
favicon: https://git.csclub.uwaterloo.ca/public/csc-propaganda/raw/branch/master/csc-logos/csc-logo.svg
palette:
- scheme: default
toggle:
icon: material/toggle-switch-off-outline
name: Switch to dark mode
- scheme: slate
toggle:
icon: material/toggle-switch
name: Switch to light mode
repo_url: https://git.csclub.uwaterloo.ca/cloud/docs
repo_name: cloud/docs
copyright: Copyright &copy; 2021 Computer Science Club of the University of Waterloo
Loading…
Cancel
Save