1830: Remove concept of under/overloaded from TemperedWMin

[cz4rs]

fixes #1830

[github-actions]

Pipelines results

PR tests (gcc-6, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-3.9, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (nvidia cuda 10.1, ubuntu, mpich)

Build for 3c8faeb

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-10, ubuntu, openmpi, no LB)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-9, ubuntu, mpich, zoltan)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-5.0, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (nvidia cuda 11.0, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-7, ubuntu, mpich, trace runtime, LB)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-9, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-13, alpine, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (intel icpx, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-8, ubuntu, mpich, address sanitizer)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-12, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-11, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-13, ubuntu, mpich)

Build for 3c8faeb

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-11, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-14, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-12, ubuntu, mpich)

Build for cf8804f (2023-02-28 21:41:36 UTC)

Compilation - successful

Testing - passed

Build log

---

PR tests (intel icpc, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (clang-10, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

PR tests (gcc-5, ubuntu, mpich)

Build for 6121bc8

Compilation - successful

Testing - passed

Build log

---

[codecov]

Codecov Report

Merging #1860 (6121bc8) into develop (893ca59) will decrease coverage by 0.54%.
The diff coverage is 84.61%.

❗ Current head 6121bc8 differs from pull request most recent head cf8804f. Consider uploading reports for the commit cf8804f to get more accurate results

@@             Coverage Diff             @@
##           develop    #1860      +/-   ##
===========================================
- Coverage    84.90%   84.37%   -0.54%     
===========================================
  Files          721      760      +39     
  Lines        25737    26811    +1074     
===========================================
+ Hits         21852    22621     +769     
- Misses        3885     4190     +305     

Impacted Files	Coverage Δ
...vt/vrt/collection/balance/temperedlb/temperedlb.cc	71.02% <70.00%> (+0.20%)	⬆️
.../vt/vrt/collection/balance/temperedlb/temperedlb.h	100.00% <100.00%> (ø)
...rt/collection/balance/temperedwmin/temperedwmin.cc	82.85% <100.00%> (+3.54%)	⬆️
...vrt/collection/balance/temperedwmin/temperedwmin.h	100.00% <100.00%> (ø)
...c/vt/vrt/collection/balance/offlinelb/offlinelb.cc	0.00% <0.00%> (-100.00%)	⬇️
src/vt/scheduler/suspended_units.h	25.00% <0.00%> (-75.00%)	⬇️
...rc/vt/vrt/collection/balance/offlinelb/offlinelb.h	0.00% <0.00%> (-75.00%)	⬇️
src/vt/context/context_attorney.cc	33.33% <0.00%> (-66.67%)	⬇️
...vt/vrt/collection/balance/lb_data_restart_reader.h	40.00% <0.00%> (-45.72%)	⬇️
src/vt/context/runnable_context/collection.impl.h	60.00% <0.00%> (-40.00%)	⬇️
... and 249 more

[nlslatt]

Within propagateIncomingSync and propagateIncomingAsync there are conditionals on isUnderloaded() that I think should be (effectively) unconditional for TemperedWMin.

You might consider renaming underloaded_ to potential_recipients_ or something like that.

There are configurations possible for TemperedLB that are not valid for TemperedWMin, such as NormByMaxExcludeIneligible for the CMF. We'll need to validate the options at some point but, for now, know that makeSufficientlyUnderloaded should never be used with TemperedWMin.

[nlslatt]

Your changes to the algorithms to remove the concept of under/overloaded look good. When doing any testing, be sure to use the default CMF NormByMax because the others will not be valid for TemperedWMin.

A good test might be that, with knowledge set to Complete, informSync and informAsync correctly gain knowledge of loads for either all ranks or all underloaded ranks depending if TemperedWMin or TemperedLB is in use. I haven't thought through how to set up such a test yet.