Persisting the worker queue and recurring jobs

[tyler-sommer]

I'd like to be able to schedule jobs to run at specific times, and for the "schedule" to exist across application restarts. The schedule itself can remain in application memory

I'm imagining a new type, probably an interface, that looks roughly like:

// A PersistentJob is a job that is designed to be performed at a scheduled time.
type PersistentJob interface {
	// Perform invokes the job.
	Perform() error

	// Clone returns a deep copy of the job.
	Clone() (PersistentJob, error)

	// JobName returns a unique name for this job.
	JobName() string

	// ScheduledFor returns a time at which this job should run.
	ScheduledFor() time.Time

	// NextScheduledTime may return a new time to schedule the job again after a successful or failed run.
	NextScheduledTime() (time.Time, bool)

	// MarshalJob returns a byte slice representation of the job.
	MarshalJob() ([]byte, error)

	// UnmarshalJob attempts to read a job from the given byte slice.
	UnmarshalJob([]byte) error
}

Each job would exist in some sort of registry, similar to CLI commands and HTTP modules. The jobs themselves would remain in memory, and written to disk (or database?) after each change.

Further thought must be put into ensuring the design handles concurrent workers, and should it handle distributed workers?

The Scheduler itself could gain an unexported extra loop() function that blocks until something is scheduled, at which point it sets a time.TImer and blocks until it reads a value from the timer's channel, then performing the work. This function would be called whenever a scheduled job is added, so potentially several blocked goroutines may spawn, but I think that's ideal.

When the application starts, it must read and unmarshal the jobs and start the timer loops for each one again.

Final note: the above interface may actually be split into two, a "provider" that actually performs the work given a Job. The provider would be registered with the scheduler, and it would contain all the marshal and unmarshal magic. But what would that look like? "Scheduler.ScheduleAt(PersistentJobName("some_job"), .....?