-
-
Notifications
You must be signed in to change notification settings - Fork 26
Description
While running the Chorus test suite under Visual Studio, I experienced random test flakiness (failures that would pass on the next run, and that passed on a different machine running the exact same code). The failures all looked like this:
Merging sally and bob...
Error: A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\Program Files\dotnet\'.
Failed to run as a self-contained app.
- The application was run as a self-contained app because 'C:\Users\vboxuser\code\chorus\output\Debug\net8.0\ChorusMerge.runtimeconfig.json' was not found.
- If this should be a framework-dependent app, add the 'C:\Users\vboxuser\code\chorus\output\Debug\net8.0\ChorusMerge.runtimeconfig.json' file and specify the appropriate framework.
merging test.lift failed!
This may be related to https://www.github.com/dotnet/runtime/issues/3569 and/or https://www.github.com/dotnet/sdk/issues/5973, except that the people reporting those issues seemed to be hitting this behavior consistently, whereas random nature of the failures I'm seeing suggests some kind of race condition in how ChorusMerge gets executed during tests (??).
In case it's useful, here is a list of tests where I've seen this happen:
- ChorusNotesFileSyncTests.ConflictFileIsCheckedIn
- SychronizerAdjunctTests.BothMadeChanges_MergeFailure_Fires_SimpleUpdate_WithTrue
- SychronizerAdjunctTests.CommitWithMergeHasCommitFileAndMergeFile
- SychronizerAdjunctTests.SendReceiveWithTrivialMergeCallsSimpleUpdate
- SynchronizerBadSituationTests.Sync_BothChangedBinaryFile_FailureReportedOneChosenSingleHead
- SynchronizerBadSituationTests.Sync_ExceptionInMergeCode_LeftWith2HeadsAndErrorOutputToProgress
- SynchronizerBadSituationTests.Sync_MergeFailure_LeavesNoChorusMergeProcessAlive
- SynchronizerBadSituationTests.Sync_MergeFailure_NoneOfTheOtherGuysFilesMakeItIntoWorkingDirectory
- SynchronizerBadSituationTests.Sync_MergeTimeoutExceeded_LeavesNoChorusMergeProcessAlive (here the error was "Timed out after waiting 1 second" so this might be something else)
- SynchronizerBadSituationTests.Sync_MergeWhenThereIsMoreThanOneHeadToMergeWith_MergesBoth
- SynchronizerBadSituationTests.Sync_RepeatedMergeFailure_WeAreLeftOnOurOwnWorkingDefault
- SyncScenarioTests.CanCollaborateOnLift
- SyncScenarioTests.CanShareConflictingChangeViaUsb
- SyncScenarioTests.TestNewVersion_SallyAndBobUpgradeButFredDelays
- SyncScenarioTests.TestNewVersion_SallyUpgradesToBobVersion
- SyncScenarioTests.TipUpdatedPostMerge
- HgMergeTests.HgMerge_LaunchesChorusMerge
All those failures were on the net8.0 framework run, after the net462 framework had passed those same tests. A different computer running the same tests passed all of the above, on both net462 and net8.0. And recompiling the Chorus project, then re-running the tests, passed those same tests that had failed earlier. So it truly does appear random whether or not this will trigger.