hyperion-supervisor monitoring of hyperion-blueapi progress + alerts for no beam and other cases where hyperion is stuck

[rtuck99]

Fixes

• Hyperion: Think of a way to do a global timeout/heartbeat for plans as well callbacks #1362

Depends on

• hyperion-supervisor cannot run load_centre_collect unable to serialize parameter_model_version #1564
  Link to dodal PR (if required):
• Add WatchableAsyncStatus support to the xbpm_feedback status dodal#1928

(remember to update pyproject.toml with the dodal commit tag if you need it for tests to pass!)

When running in blueapi mode, hyperion-supervisor now monitors the progress of the call to hyperion-blueapi.
If hyperion-blueapi makes no progress for 10 minutes:

• If hyperion-blueapi is waiting for xbpm feedback (no beam), then hyperion will raise an alert every 10 minutes until beam feedback is ok
• If hyperion-blueapi is not waiting for xbpm feedback, then hyperion will raise an error alert, abort the current collection and hand the baton back.

Instructions to reviewer on how to test:

1. Tests pass
2. Hyperion behaves as above
3. Documentation is correct

Checks for reviewer

• Would the PR title make sense to a user on a set of release notes

[codecov]

Codecov Report

❌ Patch coverage is 98.59155% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.90%. Comparing base (0039c45) to head (c5892f9).

Additional details and impacted files

@@                               Coverage Diff                                @@
##           1564_unable_to_serialize_load_centre_collect    #1633      +/-   ##
================================================================================
+ Coverage                                         92.86%   92.90%   +0.04%     
================================================================================
  Files                                               156      157       +1     
  Lines                                              8698     8768      +70     
================================================================================
+ Hits                                               8077     8146      +69     
- Misses                                              621      622       +1     

Components	Coverage Δ
i24 SSX	78.60% <ø> (ø)
hyperion	98.25% <98.59%> (+<0.01%)	⬆️
other	98.23% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.