2021-12-06 00:41:12 -05:00
|
|
|
# Merlin
|
|
|
|
|
[](https://ci.csclub.uwaterloo.ca/public/mirror)
|
|
|
|
|
|
2021-11-07 02:15:11 -05:00
|
|
|
This folder contains the code for merlin (which does the actual syncing) and arthur (which sends commands to merlin).
|
2021-09-17 00:31:49 -04:00
|
|
|
|
2022-02-09 02:37:28 -05:00
|
|
|
Check out the the [mirror env](https://git.csclub.uwaterloo.ca/public/mirror-env) for a testing environment
|
|
|
|
|
|
2022-06-22 22:43:42 -04:00
|
|
|
### Usage
|
|
|
|
|
```
|
|
|
|
|
go build merlin.go
|
|
|
|
|
```
|
|
|
|
|
Then configure `merlin-config.ini` and run using `./merlin`
|
|
|
|
|
|
2022-02-25 23:23:06 -05:00
|
|
|
### Nice Features To Add
|
|
|
|
|
- detect if an rsync process is stuck (watch the stdout/stderr of the rsync processes)
|
2022-06-22 22:43:42 -04:00
|
|
|
- get the config or the rsync commond that a repo will sync with using a cli tool
|
2022-07-02 19:35:26 -04:00
|
|
|
- improve conversion from exit status enum to string
|
|
|
|
|
- sort `arthur status` by last time synced
|
2022-07-15 00:31:43 -04:00
|
|
|
- last sync runtime, time until next sync
|
|
|
|
|
- when sync fails, then make a copy of the rsync logs
|
2021-11-01 23:21:45 -04:00
|
|
|
|
|
|
|
|
### Completed
|
2022-02-25 23:23:06 -05:00
|
|
|
- [x] add bwlimit option for each rsync process
|
2021-11-01 23:21:45 -04:00
|
|
|
- [x] write process manager
|
|
|
|
|
- [x] save state (last attempted time, last attempted status) for each repo, and restore state on startup (e.g. use JSON/INI file for each repo)
|
|
|
|
|
- [x] calculate difference between the scheduled time of a job and the time at which it actually ran; log this
|
|
|
|
|
- [x] add all repos to merlin-config.ini (\*)
|
2021-11-05 00:17:55 -04:00
|
|
|
- [x] handle termination signals in merlin (SIGINT, SIGTERM); close stopChan for this
|
2021-11-07 02:01:58 -05:00
|
|
|
- [x] listen on Unix socket in merlin
|
|
|
|
|
- [x] implement arthur.go (commands: sync and status)
|
2021-11-14 17:22:32 -05:00
|
|
|
- [x] allow dynamic reloading in merlin
|
|
|
|
|
- [x] use separate log file for each child process (currently sharing stdout/stderr with parent)
|
|
|
|
|
- [x] implement zfssync in merlin (just invoke the existing Python script)
|
