job: Manage job lifecycles individually instead of as groups

[joamaki]

The job scheduler used to register entire job.Groups as lifecycle hooks via module-private providers (cilium#pkg/hive/hive.go). That meant a group (and any “dynamic” jobs attached to it) could be started before all of its dependencies were known, so a goroutine might run prior to the component it relies on. Or a job might be stopped before other jobs depending on it were stopped.

For example:

  cell.Module(...,
    cell.Invoke(func(jg job.Group, a A) { jg.Add(... use A ...) },
    cell.Invoke(func(jg job.Group, b B) { jg.Add(... use B ...) },
  )

The jg is constructed by the first invoke and this causes it to be appended to the hive lifecycle.
Assuming B wasn't constructed yet, the second invoke will construct it and potentially its
constructor adds Bs start hooks to lifecycle. Now jg will start after A's start hooks but
before Bs start hooks. This means there's a chance that the job that uses B will execute before
B's start hook is called.

The refactor fixes that by:

• Teaching cell.DefaultLifecycle about a HookDescriptiveInterface, so lifecycle logs/PrintHooks can include job names.
• Letting Lifecycle.Start run again to start newly appended hooks; the registry now appends dynamic jobs after Hive startup and immediately calls Start to bring them online.
• Moving lifecycle ownership out of individual groups and into the registry: the registry now holds the lifecycles and manages indvidual jobs. Groups are lightweight containers forwarding Add calls to the registry, so jobs added before Hive start still get queued, while jobs added later are started immediately but shut down in reverse order when the registry stops.
• Updating all job implementations/tests and call sites to the simplified API, plus adding a log-driven test (TestJobLifecycleOrderingAcrossGroups) that asserts jobs start in the order added (even across multiple groups) and stop in reverse order, covering both jobs added before starting and after.

Together these changes eliminate the race where a job could start before its dependency, while also giving more precise lifecycle logging.

[pippolo84]

LGTM, just a style nit left inline. 💯

[dylandreimerink]

Neat, I like re-using the lifecycles instead of the custom implementation, nice cleanup.