# Merlin
[![Build Status](https://ci.csclub.uwaterloo.ca/api/badges/public/mirror/status.svg)](https://ci.csclub.uwaterloo.ca/public/mirror)

This folder contains the code for merlin (which does the actual syncing) and arthur (which sends commands to merlin).

Check out the the [mirror env](https://git.csclub.uwaterloo.ca/public/mirror-env) for a testing environment

### Usage
```
go build merlin.go
```
Then configure `merlin-config.ini` and run using `./merlin`

### Nice Features To Add
- detect if an rsync process is stuck (watch the stdout/stderr of the rsync processes)
- detect if config file is changed and automatically reload ([fsnotify](https://github.com/fsnotify/fsnotify))
- sort `arthur status` by last time synced
- respond to `arthur` without blocking main thread
- state files appear to be getting extra newlines
- log rotation for log files (also maybe state reporting to prometheus)
- split off arthur with a more featureful arthur for debugging
    - last sync runtime, time until next sync, due to sync (true/false for repo would sync if could)
    - get expected rsync command or repo config
    - details for last 10 syncs (avg time, success rate, data read/written)

### Completed
- [x] add bwlimit option for each rsync process
- [x] write process manager
- [x] save state (last attempted time, last attempted status) for each repo, and restore state on startup (e.g. use JSON/INI file for each repo)
- [x] calculate difference between the scheduled time of a job and the time at which it actually ran; log this
- [x] add all repos to merlin-config.ini (\*)
- [x] handle termination signals in merlin (SIGINT, SIGTERM); close stopChan for this
- [x] listen on Unix socket in merlin
- [x] implement arthur.go (commands: sync and status)
- [x] allow dynamic reloading in merlin
- [x] use separate log file for each child process (currently sharing stdout/stderr with parent)
- [x] implement zfssync in merlin (just invoke the existing Python script)
