204 lines
6.7 KiB
Markdown
204 lines
6.7 KiB
Markdown
# pyceo
|
|
[](https://ci.csclub.uwaterloo.ca/public/pyceo)
|
|
|
|
CEO (**C**SC **E**lectronic **O**ffice) is the tool used by CSC to manage
|
|
club accounts and memberships. See [architecture.md](architecture.md) for an
|
|
overview of its architecture.
|
|
|
|
## Development
|
|
### Docker
|
|
If you are not modifying code related to email or Mailman, then you may use
|
|
Docker containers instead, which are much easier to work with than the VM.
|
|
```
|
|
docker.sh up
|
|
```
|
|
This will create some containers with the bare minimum necessary for ceod to
|
|
run. Run `docker logs -f phosphoric-acid` and wait until you see the line
|
|
`sleep infinity`. Then attach to each of phosphoric-acid, mail and coffee,
|
|
and start ceod (see 'Running the application', below). Once inside a container,
|
|
make sure to `cd` into the current working directory on the host.
|
|
|
|
To use ceo, run the following inside the phosphoric-acid container:
|
|
```
|
|
login
|
|
<username is ctdalek, password is krb5>
|
|
<cd into your directory OUTSIDE the container>
|
|
. venv/bin/activate
|
|
python -m ceo
|
|
```
|
|
This should bring up the TUI.
|
|
|
|
### VM
|
|
If you are making changes related to email or Mailman, you will need the full
|
|
[syscom dev environment](https://git.uwaterloo.ca/csc/syscom-dev-environment).
|
|
This will setup all of the services needed for ceo to work. You should clone
|
|
this repo in the phosphoric-acid container under ctdalek's home directory; you
|
|
will then be able to access it from any container thanks to NFS.
|
|
|
|
Once you have the dev environment setup, there are a few more steps you'll
|
|
need to do for ceo.
|
|
|
|
#### Kerberos principals
|
|
First, you'll need `ceod/<hostname>` principals for each of phosphoric-acid,
|
|
coffee and mail. (coffee is taking over the role of caffeine for the DB
|
|
endpoints). For example, in the phosphoric-acid container:
|
|
```sh
|
|
kadmin -p sysadmin/admin
|
|
<password is krb5>
|
|
addprinc -randkey ceod/phosphoric-acid.csclub.internal
|
|
ktadd ceod/phosphoric-acid.csclub.internal
|
|
```
|
|
Do this for coffee and mail as well. You need to actually be in the
|
|
appropriate container when running these commands, since the credentials
|
|
are being added to the local keytab.
|
|
On phosphoric-acid, you will additionally need to create a principal
|
|
called `ceod/admin` (remember to addprinc **and** ktadd).
|
|
|
|
#### Database
|
|
**Note**: The instructions below apply to the dev environment only; in
|
|
production, the DB superusers should be restricted to the host where
|
|
the DB is running.
|
|
|
|
Attach to the coffee container, run `mysql`, and run the following:
|
|
|
|
```
|
|
CREATE USER 'mysql' IDENTIFIED BY 'mysql';
|
|
GRANT ALL PRIVILEGES ON *.* TO 'mysql' WITH GRANT OPTION;
|
|
```
|
|
(In prod, the superuser should have '@localhost' appended to its name.)
|
|
|
|
Now open /etc/mysql/mariadb.conf.d/50-server.cnf and comment out the following line:
|
|
```
|
|
bind-address = 127.0.0.1
|
|
```
|
|
Then restart MariaDB:
|
|
```
|
|
systemctl restart mariadb
|
|
```
|
|
|
|
Install PostgreSQL in the container:
|
|
```
|
|
apt install -y postgresql
|
|
```
|
|
Modify the superuser `postgres` for password authentication and restrict new users:
|
|
```
|
|
su postgres
|
|
psql
|
|
|
|
ALTER USER postgres WITH PASSWORD 'postgres';
|
|
REVOKE ALL ON SCHEMA public FROM public;
|
|
GRANT ALL ON SCHEMA public TO postgres;
|
|
```
|
|
Create a new `pg_hba.conf`:
|
|
```
|
|
cd /etc/postgresql/<version>/<branch>/
|
|
mv pg_hba.conf pg_hba.conf.old
|
|
```
|
|
```
|
|
# new pg_hba.conf
|
|
# TYPE DATABASE USER ADDRESS METHOD
|
|
local all postgres peer
|
|
host all postgres 0.0.0.0/0 md5
|
|
|
|
local all all peer
|
|
host all all localhost md5
|
|
|
|
local sameuser all md5
|
|
host sameuser all 0.0.0.0/0 md5
|
|
```
|
|
**Warning**: in prod, the postgres user should only be allowed to connect locally,
|
|
so the relevant snippet in pg_hba.conf should look something like
|
|
```
|
|
local all postgres md5
|
|
host all postgres localhost md5
|
|
host all postgres 0.0.0.0/0 reject
|
|
host all postgres ::/0 reject
|
|
```
|
|
Add the following to postgresql.conf:
|
|
```
|
|
listen_addresses = '*'
|
|
```
|
|
Now restart PostgreSQL:
|
|
```
|
|
systemctl restart postgresql
|
|
```
|
|
**In prod**, users can login remotely but superusers (`postgres` and `mysql`) are only
|
|
allowed to login from the database host.
|
|
|
|
#### Mailman
|
|
You should create the following mailing lists from the mail container:
|
|
```sh
|
|
/opt/mailman3/bin/mailman create syscom@csclub.internal
|
|
/opt/mailman3/bin/mailman create syscom-alerts@csclub.internal
|
|
/opt/mailman3/bin/mailman create exec@csclub.internal
|
|
/opt/mailman3/bin/mailman create ceo@csclub.internal
|
|
```
|
|
See https://git.uwaterloo.ca/csc/syscom-dev-environment/-/tree/master/mail
|
|
for instructions on how to access the Mailman UI from your browser.
|
|
|
|
If you want to actually see the archived messages, you'll
|
|
need to tweak the settings for each list from the UI so that non-member
|
|
messages get accepted (by default they get held).
|
|
|
|
|
|
#### Dependencies
|
|
Next, install and activate a virtualenv:
|
|
```sh
|
|
sudo apt install libkrb5-dev libpq-dev python3-dev
|
|
python3 -m venv venv
|
|
. venv/bin/activate
|
|
pip install -r requirements.txt
|
|
pip install -r dev-requirements.txt
|
|
```
|
|
|
|
## Running the application
|
|
ceod is a distributed application, with instances on different hosts offering
|
|
different services.
|
|
Therefore, you will need to run ceod on multiple hosts. Currently, those are
|
|
phosphoric-acid, mail and caffeine (in the dev environment, caffeine is
|
|
replaced by coffee).
|
|
|
|
To run ceod on a single host (as root, since the app needs to read the keytab):
|
|
```sh
|
|
export FLASK_APP=ceod.api
|
|
export FLASK_ENV=development
|
|
flask run -h 0.0.0.0 -p 9987
|
|
```
|
|
|
|
Sometimes changes you make in the source code don't show up while Flask
|
|
is running. Stop the flask app (Ctrl-C), run `clear_cache.sh`, then
|
|
restart the app.
|
|
|
|
## Interacting with the application
|
|
To use the TUI:
|
|
```
|
|
python -m ceo
|
|
```
|
|
To use the CLI:
|
|
```
|
|
python -m ceo --help
|
|
```
|
|
|
|
Alternatively, you may use curl to send HTTP requests.
|
|
|
|
ceod uses [SPNEGO](https://en.wikipedia.org/wiki/SPNEGO) for authentication,
|
|
and TLS for confidentiality and integrity. In development mode, TLS can be
|
|
disabled.
|
|
|
|
First, make sure that your version of curl has been compiled with SPNEGO
|
|
support:
|
|
```sh
|
|
curl -V
|
|
```
|
|
Your should see 'SPNEGO' in the 'Features' section.
|
|
|
|
Here's an example of making a request to an endpoint which writes to LDAP:
|
|
```sh
|
|
# Get a Kerberos TGT first
|
|
kinit
|
|
# Make the request
|
|
curl --negotiate -u : --service-name ceod --delegation always \
|
|
-d '{"uid":"test_1","cn":"Test One","program":"Math","terms":["s2021"]}' \
|
|
-X POST http://phosphoric-acid:9987/api/members
|
|
```
|