I think it would be useful for CSC members to access their PostgreSQL database from any CSC machine. (We should also do the same for MySQL at some point).
I think it would be useful for CSC members to access their PostgreSQL database from any CSC machine. (We should also do the same for MySQL at some point).
I think so.
For MySQL, I think it should be possible to allow connections for users from a specific subnet: https://stackoverflow.com/questions/11742963/how-to-grant-remote-access-to-mysql-for-a-whole-subnet/38389851#38389851
Hmm now that I look into it I think filtering by subnet is a bit iffy.
mysql's config will be in pyceo while psql's subnet config must be set in pg_hba.conf
mysql does not accept CIDR subnet notation (need to use 192.168.100.% or 192.168.100.0/255.255.255.0)
need to add extra condition for localhost and 127.0.0.1 (overwise MySQL will try to create the same user twice)
Hmm now that I look into it I think filtering by subnet is a bit iffy.
- mysql's config will be in pyceo while psql's subnet config must be set in `pg_hba.conf`
- mysql does not accept CIDR subnet notation (need to use `192.168.100.%` or `192.168.100.0/255.255.255.0`)
- need to add extra condition for localhost and `127.0.0.1` (overwise MySQL will try to create the same user twice)
I don't think this is the right type of error to raise here. Should probably be something like 'InvalidUsername'.
Also, you want to raise an instance of the Exception class, not the class itself:
raiseInvalidUsername()
I don't think this is the right type of error to raise here. Should probably be something like 'InvalidUsername'.
Also, you want to raise an instance of the Exception class, not the class itself:
```python
raise InvalidUsername()
```
I'm not sure if this is intuitive behaviour - if the user wants to reset their database password, there should be a separate endpoint for that.
Maybe something like POST /api/db/mysql/<username>/pwreset.
I'm not sure if this is intuitive behaviour - if the user wants to reset their database password, there should be a separate endpoint for that.
Maybe something like `POST /api/db/mysql/<username>/pwreset`.
On second thought, I don't think it's necessary to implement password reset behaviour, since the old ceo didn't have it.
The password will be written to a file in their home directory anyways, so they don't really have an excuse to lose it.
On second thought, I don't think it's necessary to implement password reset behaviour, since the old ceo didn't have it.
The password will be written to a file in their home directory anyways, so they don't really have an excuse to lose it.
The old ceo would reset the database password if you already had a database and tried to create a mysql database.
I was kind of basing it off that when I wrote this.
The old ceo would reset the database password if you already had a database and tried to create a mysql database.
I was kind of basing it off that when I wrote this.
I see. I still think this is non-intuitive behaviour, to be honest. If I clicked a button titled "create database", and it reset my DB password instead, I would be pretty confused.
I think we should just return an error here. If we really want to, we can add an extra endpoint for resetting DB passwords, but this should happen very rarely. Up to you.
I see. I still think this is non-intuitive behaviour, to be honest. If I clicked a button titled "create database", and it reset my DB password instead, I would be pretty confused.
I think we should just return an error here. If we really want to, we can add an extra endpoint for resetting DB passwords, but this should happen very rarely. Up to you.
Ok I'll return a UserAlreadyExistsError (as in mysql/psql user already exists) and add the password reset endpoint since I wrote most of the code already.
Ok I'll return a UserAlreadyExistsError (as in mysql/psql user already exists) and add the password reset endpoint since I wrote most of the code already.
I think it would be better if it just did nothing because it is possible for the mysql users to drop their database (and recreate it themselves)
But for postgresql if someone somehow dropped their database they wouldn't be able to even login
I think it would be better if it just did nothing because it is possible for the mysql users to drop their database (and recreate it themselves)
But for postgresql if someone somehow dropped their database they wouldn't be able to even login
I've haven't added unit tests because the api maps pretty directly to the methods but I'm not sure if they work since the build has always been failing on this branch.
Also do I need to add tests to check for handling of invalid configs?
I've haven't added unit tests because the api maps pretty directly to the methods but I'm not sure if they work since the build has always been failing on this branch.
Also do I need to add tests to check for handling of invalid configs?
I've haven't added unit tests because the api maps pretty directly to the methods but I'm not sure if they work since the build has always been failing on this branch.
We'll need to add a service container to Drone for the database. I'll try to take care of it this weekend.
Also, you appear to be failing the lint stage: https://ci.csclub.uwaterloo.ca/public/pyceo/41/1/3
Also do I need to add tests to check for handling of invalid configs?
No, I don't think so. Although it would be a good idea to check for an invalid config at startup time.
> I've haven't added unit tests because the api maps pretty directly to the methods but I'm not sure if they work since the build has always been failing on this branch.
We'll need to add a service container to Drone for the database. I'll try to take care of it this weekend.
Also, you appear to be failing the lint stage: https://ci.csclub.uwaterloo.ca/public/pyceo/41/1/3
> Also do I need to add tests to check for handling of invalid configs?
No, I don't think so. Although it would be a good idea to check for an invalid config at startup time.
@a268wang since we can't guarantee that caffeine will start up first on the real CSC machines, I decided to take out the initial DB check at startup. Sorry for making you do extra work.
@a268wang since we can't guarantee that caffeine will start up first on the real CSC machines, I decided to take out the initial DB check at startup. Sorry for making you do extra work.
merenber
May need to install libpq-dev and/or postgresql-server-dev-11 for psycopg2
Excellent work. Just a few minor adjustments, and unit tests.
REVOKE ALL ON SCHEMA public FROM public;GRANT ALL ON SCHEMA public TO postgres;```create a new `pg_hba.conf` to force password authentication and reject non localI think it would be useful for CSC members to access their PostgreSQL database from any CSC machine. (We should also do the same for MySQL at some point).
Will it be fine if I just change the postgres config to allow connections from any host?
I think so.
For MySQL, I think it should be possible to allow connections for users from a specific subnet: https://stackoverflow.com/questions/11742963/how-to-grant-remote-access-to-mysql-for-a-whole-subnet/38389851#38389851
I've changed both mysql and postgres to allow connections from any source.
Is there a subnet that you want me to use instead?
129.97.134.0/24, 2620:101:f000:4901::/64 (MC VLAN 134)
This should be configurable from ceod.ini.
For the dev environment, use 192.168.100.0/24.
Hmm now that I look into it I think filtering by subnet is a bit iffy.
pg_hba.conf192.168.100.%or192.168.100.0/255.255.255.0)127.0.0.1(overwise MySQL will try to create the same user twice)Alright, in that case, don't bother. Just let members connect from any IP address.
def create_db_from_type(db_type: str, username: str):try:if not username.isalnum(): # username should not contain symbolsraise UserNotFoundErrorI don't think this is the right type of error to raise here. Should probably be something like 'InvalidUsername'.
Also, you want to raise an instance of the Exception class, not the class itself:
except DatabasePermissionError:return {'error': 'unable to perform action due to permissions'}, 502except:return {'error': 'Unexpected error'}, 500You don't need this last case - unhandled exceptions are handled in error_handlers.py.
@implementer(IDatabaseService)class MySQLService:def __init__(self):self.type = 'mysql'I suggest creating a class variable, like so:
password = gen_password()search_for_user = f"SELECT user FROM mysql.user WHERE user='{username}'"search_for_db = f"SHOW DATABASES LIKE '{username}'"create_user = f"CREATE USER '{username}'@'localhost' IDENTIFIED BY %(password)s"I think we should allow CSC members to access their database from any CSC machine.
if response_is_empty(search_for_user, con):cursor.execute(create_user, {'password': password})else:cursor.execute(reset_password, {'password': password})I'm not sure if this is intuitive behaviour - if the user wants to reset their database password, there should be a separate endpoint for that.
Maybe something like
POST /api/db/mysql/<username>/pwreset.On second thought, I don't think it's necessary to implement password reset behaviour, since the old ceo didn't have it.
The password will be written to a file in their home directory anyways, so they don't really have an excuse to lose it.
The old ceo would reset the database password if you already had a database and tried to create a mysql database.
I was kind of basing it off that when I wrote this.
I see. I still think this is non-intuitive behaviour, to be honest. If I clicked a button titled "create database", and it reset my DB password instead, I would be pretty confused.
I think we should just return an error here. If we really want to, we can add an extra endpoint for resetting DB passwords, but this should happen very rarely. Up to you.
Ok I'll return a UserAlreadyExistsError (as in mysql/psql user already exists) and add the password reset endpoint since I wrote most of the code already.
else:cursor.execute(reset_password, {'password': password})if response_is_empty(search_for_db, con):cursor.execute(create_database)If the user already exists but somehow their database doesn't, we should return an error.
I think it would be better if it just did nothing because it is possible for the mysql users to drop their database (and recreate it themselves)
But for postgresql if someone somehow dropped their database they wouldn't be able to even login
Sounds good.
) as con:with con.cursor() as cursor:cursor.execute(drop_user)cursor.execute(drop_db)Not that it matters, but doesn't it make more sense to drop the database before dropping the user? Since the user logically "owns" the database?
if response_is_empty(search_for_user, con):cursor.execute(create_user, {'password': password})else:cursor.execute(reset_password, {'password': password})Same comments as for MySQL.
I've haven't added unit tests because the api maps pretty directly to the methods but I'm not sure if they work since the build has always been failing on this branch.
Also do I need to add tests to check for handling of invalid configs?
We'll need to add a service container to Drone for the database. I'll try to take care of it this weekend.
Also, you appear to be failing the lint stage: https://ci.csclub.uwaterloo.ca/public/pyceo/41/1/3
No, I don't think so. Although it would be a good idea to check for an invalid config at startup time.
@a268wang since we can't guarantee that caffeine will start up first on the real CSC machines, I decided to take out the initial DB check at startup. Sorry for making you do extra work.
eb5d632606into v1 2 years agoReviewers
eb5d632606.