Merlin

This folder contains the code for merlin (which does the actual syncing) and arthur (which sends commands to merlin).

Check out the the mirror env for a testing environment

go build merlin.go

Then configure merlin-config.ini and run using ./merlin

detect if an rsync process is stuck (watch the stdout/stderr of the rsync processes)
detect if config file is changed and automatically reload (fsnotify)
sort arthur status by last time synced
respond to arthur without blocking main thread
state files appear to be getting extra newlines
log rotation for log files (also maybe state reporting to prometheus)
split off arthur with a more featureful arthur for debugging
- last sync runtime, time until next sync
- get expected rsync command or repo config
- details for last 10 syncs (avg time, success rate, data read/written)

add bwlimit option for each rsync process
write process manager
save state (last attempted time, last attempted status) for each repo, and restore state on startup (e.g. use JSON/INI file for each repo)
calculate difference between the scheduled time of a job and the time at which it actually ran; log this
add all repos to merlin-config.ini (*)
handle termination signals in merlin (SIGINT, SIGTERM); close stopChan for this
listen on Unix socket in merlin
implement arthur.go (commands: sync and status)
allow dynamic reloading in merlin
use separate log file for each child process (currently sharing stdout/stderr with parent)
implement zfssync in merlin (just invoke the existing Python script)