Apply fix for quick network reconnect after backend destroy #1193

marmarek · 2025-10-10T01:37:33Z

This especially is relevant for paused VMs (like preloaded disposables).
If backend was restarted while frontend was paused, qubesd will attempt
to reconnect it on unpause, which will trigger the race condition.

qubesos-bot · 2025-10-10T12:26:43Z

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025103005-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025081011-4.3&flavor=update

system_tests_extra
- TC_00_QVCTest_debian-13-xfce: test_010_screenshare (failure + cleanup)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1179648 != 0
- TC_00_QVCTest_fedora-42-xfce: test_010_screenshare (failure + cleanup)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1179648 != 0
- TC_00_QVCTest_whonix-gateway-18: test_010_screenshare (failure)
  AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...
- TC_00_QVCTest_whonix-workstation-18: test_010_screenshare (failure)
  AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...
system_tests_basic_vm_qrexec_gui_zfs
- switch_pool: Failed (test died)
  # Test died: command 'modprobe zfs zfs_arc_max=67108864' failed at ...
system_tests_qwt_win10_seamless@hw13
- windows_install: Failed (test died)
  # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...
system_tests_qwt_win11@hw13
- windows_install: wait_serial (wait serial expected)
  # wait_serial expected: qr/iDVvW-\d+-/...
- windows_install: Failed (test died + timed out)
  # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...
system_tests_gui_interactive_preloaded
- collect_logs: wait_serial (wait serial expected)
  # wait_serial expected: qr/zhGWq-\d+-/...
- collect_logs: Failed (test died + timed out)
  # Test died: command 'curl --form upload=@journalctl.log --form upn...
- collect_logs: wait_serial (wait serial expected)
  # wait_serial expected: qr/Dw02q-\d+-/...
system_tests_dispvm_perf@hw7
- TC_00_DispVMPerf_whonix-workstation-18: test_006_dom0_dispvm (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-18: test_026_dom0_dispvm_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
system_tests_basic_vm_qrexec_gui_xfs
- TC_20_NonAudio_fedora-42-xfce-pool: test_000_start_shutdown (error + timed out)
  qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
system_tests_whonix@hw7
- whonixcheck: fail (unknown)
  Whonixcheck for anon-whonix failed...
- whonixcheck: fail (unknown)
  Whonixcheck for whonix-gateway-18 failed...
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: fail (unknown)
  Whonixcheck for whonix-workstation-18 failed...
- whonixcheck: Failed (test died)
  # Test died: systemcheck failed at qubesos/tests/whonixcheck.pm lin...

Failed tests

19 failures

system_tests_extra
- TC_00_QVCTest_debian-13-xfce: test_010_screenshare (failure + cleanup)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1179648 != 0
- TC_00_QVCTest_fedora-42-xfce: test_010_screenshare (failure + cleanup)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1179648 != 0
- TC_00_QVCTest_whonix-gateway-18: test_010_screenshare (failure)
  AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...
- TC_00_QVCTest_whonix-workstation-18: test_010_screenshare (failure)
  AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...
system_tests_basic_vm_qrexec_gui_zfs
- switch_pool: Failed (test died)
  # Test died: command 'modprobe zfs zfs_arc_max=67108864' failed at ...
system_tests_qwt_win10_seamless@hw13
- windows_install: Failed (test died)
  # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...
system_tests_qwt_win11@hw13
- windows_install: wait_serial (wait serial expected)
  # wait_serial expected: qr/iDVvW-\d+-/...
- windows_install: Failed (test died + timed out)
  # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...
system_tests_gui_interactive_preloaded
- collect_logs: wait_serial (wait serial expected)
  # wait_serial expected: qr/zhGWq-\d+-/...
- collect_logs: Failed (test died + timed out)
  # Test died: command 'curl --form upload=@journalctl.log --form upn...
- collect_logs: wait_serial (wait serial expected)
  # wait_serial expected: qr/Dw02q-\d+-/...
system_tests_dispvm_perf@hw7
- TC_00_DispVMPerf_whonix-workstation-18: test_006_dom0_dispvm (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-18: test_026_dom0_dispvm_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
system_tests_basic_vm_qrexec_gui_xfs
- TC_20_NonAudio_fedora-42-xfce-pool: test_000_start_shutdown (error + timed out)
  qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
system_tests_whonix@hw7
- whonixcheck: fail (unknown)
  Whonixcheck for anon-whonix failed...
- whonixcheck: fail (unknown)
  Whonixcheck for whonix-gateway-18 failed...
- whonixcheck: fail (unknown)
  Whonixcheck for sys-whonix failed...
- whonixcheck: fail (unknown)
  Whonixcheck for whonix-workstation-18 failed...
- whonixcheck: Failed (test died)
  # Test died: systemcheck failed at qubesos/tests/whonixcheck.pm lin...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/149225#dependencies

82 fixed

system_tests_extra
- TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
  AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...
system_tests_dispvm
- system_tests: Fail (unknown)
  Tests qubes.tests.integ.dispvm failed (exit code 1), details report...
- TC_20_DispVM_debian-13-xfce: test_012_preload_low_mem (failure)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1 != 0
- TC_20_DispVM_debian-13-xfce: test_013_preload_gui (error)
  raise KeyError(key)... KeyError: 'disp3723'
- TC_20_DispVM_debian-13-xfce: test_014_preload_nogui (error + cleanup)
  raise TimeoutError from exc_val... TimeoutError
- TC_20_DispVM_debian-13-xfce: test_015_preload_race_more (error + cleanup)
  raise KeyError(key)... KeyError: 'disp1187'
- TC_20_DispVM_debian-13-xfce: test_016_preload_race_less (failure + cleanup)
  ^^^^^^^^^^^^^^^^^^^^^^... AssertionError
- TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error)
  raise KeyError(key)... KeyError: 'disp7317'
- TC_20_DispVM_debian-13-xfce: test_018_preload_global (error)
  raise KeyError(key)... KeyError: 'disp8572'
- TC_20_DispVM_debian-13-xfce: test_019_preload_refresh (error)
  raise KeyError(key)... KeyError: 'disp6425'
- TC_20_DispVM_fedora-42-xfce: test_012_preload_low_mem (failure)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1 != 0
- TC_20_DispVM_whonix-workstation-17: test_012_preload_low_mem (failure)
  ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError: 1 != 0
system_tests_kde_gui_interactive
- gui_keyboard_layout: wait_serial (wait serial expected)
  # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...
- gui_keyboard_layout: Failed (test died)
  # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...
system_tests_audio
- system_tests: Fail (unknown)
  Tests qubes.tests.integ.audio failed (exit code 1), details reporte...
- system_tests: Failed (test died)
  # Test died: Some tests failed at qubesos/tests/system_tests.pm lin...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_223_audio_play_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_224_audio_rec_muted_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_225_audio_rec_unmuted_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_252_audio_playback_audiovm_switch_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
system_tests_audio@hw1
- system_tests: Fail (unknown)
  Tests qubes.tests.integ.audio failed (exit code 1), details reporte...
- system_tests: Failed (test died)
  # Test died: Some tests failed at qubesos/tests/system_tests.pm lin...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_223_audio_play_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 60 secon...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_224_audio_rec_muted_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 60 secon...
- TC_20_AudioVM_Pulse_whonix-workstation-17: test_252_audio_playback_audiovm_switch_hvm (error)
  qubes.exc.QubesVMError: Cannot connect to qrexec agent for 60 secon...
system_tests_dispvm_perf@hw7
- system_tests: wait_serial (wait serial expected)
  # wait_serial expected: qr/D2qNh-\d+-/...
- TC_00_DispVMPerf_debian-13-xfce: test_000_dispvm (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_002_dispvm_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_006_dispvm_from_dom0 (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_008_dispvm_from_dom0_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_020_dispvm_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_021_dispvm_preload_gui (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_022_dispvm_preload_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_023_dispvm_preload_gui_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_026_dispvm_from_dom0_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_027_dispvm_from_dom0_preload_gui (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_028_dispvm_from_dom0_preload_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_029_dispvm_from_dom0_preload_gui_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_400_dispvm_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_402_dispvm_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_404_dispvm_preload_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_404_dispvm_preload_less_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_404_dispvm_preload_more_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_405_dispvm_preload_gui_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_406_dispvm_preload_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_407_dispvm_preload_gui_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_900_vm (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_902_vm_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_904_vm_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_debian-13-xfce: test_906_vm_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_020_dispvm_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_021_dispvm_preload_gui (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_022_dispvm_preload_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_023_dispvm_preload_gui_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_026_dispvm_from_dom0_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_027_dispvm_from_dom0_preload_gui (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_028_dispvm_from_dom0_preload_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_029_dispvm_from_dom0_preload_gui_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_404_dispvm_preload_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_404_dispvm_preload_less_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_404_dispvm_preload_more_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_405_dispvm_preload_gui_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_406_dispvm_preload_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_fedora-42-xfce: test_407_dispvm_preload_gui_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_020_dispvm_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_021_dispvm_preload_gui (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_022_dispvm_preload_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_023_dispvm_preload_gui_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_026_dispvm_from_dom0_preload (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_027_dispvm_from_dom0_preload_gui (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_028_dispvm_from_dom0_preload_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_029_dispvm_from_dom0_preload_gui_concurrent (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_404_dispvm_preload_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_404_dispvm_preload_less_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_404_dispvm_preload_more_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_405_dispvm_preload_gui_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_406_dispvm_preload_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
- TC_00_DispVMPerf_whonix-workstation-17: test_407_dispvm_preload_gui_concurrent_api (failure)
  AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...
system_tests_guivm_gpu_gui_interactive@hw13
- guivm_startup: wait_serial (wait serial expected)
  # wait_serial expected: qr/lEcbc-\d+-/...
- guivm_startup: Failed (test died + timed out)
  # Test died: command '! qvm-check sys-whonix || time qvm-start sys-...
system_tests_basic_vm_qrexec_gui_ext4
- system_tests: Fail (unknown)
  Tests qubes.tests.integ.vm_qrexec_gui failed (exit code 1), details...
- TC_20_NonAudio_whonix-gateway-17-pool: test_012_qubes_desktop_run (error + cleanup)
  raise TimeoutError from exc_val... TimeoutError

Unstable tests

Details

Performance Tests

Performance degradation:

13 performance degradations

dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 163913.00 🔻 ( previous job: 497426.00, degradation: 32.95%)
dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 93576.00 🔻 ( previous job: 265260.00, degradation: 35.28%)
dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 253462.00 🔻 ( previous job: 431512.00, degradation: 58.74%)
dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 74990.00 🔻 ( previous job: 196254.00, degradation: 38.21%)
dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 13606.00 🔻 ( previous job: 23940.00, degradation: 56.83%)
fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2319.00 🔻 ( previous job: 3020.00, degradation: 76.79%)
fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1125.00 🔻 ( previous job: 1368.00, degradation: 82.24%)
fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 2187.00 🔻 ( previous job: 3765.00, degradation: 58.09%)
fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 461.00 🔻 ( previous job: 1251.00, degradation: 36.85%)
fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 49371.00 🔻 ( previous job: 157382.00, degradation: 31.37%)
fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 54705.00 🔻 ( previous job: 64217.00, degradation: 85.19%)
fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2279.00 🔻 ( previous job: 4098.00, degradation: 55.61%)
fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 535.00 🔻 ( previous job: 2384.00, degradation: 22.44%)

Remaining performance tests:

165 tests

debian-13-xfce_exec: 6.80 🟢 ( previous job: 8.36, improvement: 81.33%)
debian-13-xfce_exec-root: 25.94 🟢 ( previous job: 27.36, improvement: 94.79%)
debian-13-xfce_socket: 8.17 🟢 ( previous job: 8.57, improvement: 95.32%)
debian-13-xfce_socket-root: 8.44 🔻 ( previous job: 8.26, degradation: 102.24%)
debian-13-xfce_exec-data-simplex: 65.15 🟢 ( previous job: 72.43, improvement: 89.95%)
debian-13-xfce_exec-data-duplex: 65.06 🟢 ( previous job: 76.65, improvement: 84.88%)
debian-13-xfce_exec-data-duplex-root: 68.92 🟢 ( previous job: 91.79, improvement: 75.08%)
debian-13-xfce_socket-data-duplex: 86.38 🟢 ( previous job: 133.45, improvement: 64.73%)
fedora-42-xfce_exec: 8.97 🟢 ( previous job: 9.06, improvement: 98.98%)
fedora-42-xfce_exec-root: 57.68 🟢 ( previous job: 58.19, improvement: 99.13%)
fedora-42-xfce_socket: 8.50 🔻 ( previous job: 8.48, degradation: 100.24%)
fedora-42-xfce_socket-root: 8.00 🟢 ( previous job: 8.18, improvement: 97.75%)
fedora-42-xfce_exec-data-simplex: 66.24 🟢 ( previous job: 78.48, improvement: 84.40%)
fedora-42-xfce_exec-data-duplex: 68.92 🔻 ( previous job: 67.92, degradation: 101.48%)
fedora-42-xfce_exec-data-duplex-root: 89.52 🟢 ( previous job: 96.36, improvement: 92.90%)
fedora-42-xfce_socket-data-duplex: 90.26 🟢 ( previous job: 142.58, improvement: 63.30%)
whonix-gateway-18_exec: 6.51
whonix-gateway-18_exec-root: 128.17
whonix-gateway-18_socket: 7.69
whonix-gateway-18_socket-root: 7.94
whonix-gateway-18_exec-data-simplex: 64.75
whonix-gateway-18_exec-data-duplex: 71.21
whonix-gateway-18_exec-data-duplex-root: 127.25
whonix-gateway-18_socket-data-duplex: 99.12
whonix-workstation-18_exec: 8.69
whonix-workstation-18_exec-root: 132.79
whonix-workstation-18_socket: 8.89
whonix-workstation-18_socket-root: 8.11
whonix-workstation-18_exec-data-simplex: 64.76
whonix-workstation-18_exec-data-duplex: 68.53
whonix-workstation-18_exec-data-duplex-root: 125.16
whonix-workstation-18_socket-data-duplex: 86.41
dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2427.00 🔻 ( previous job: 2446.00, degradation: 99.22%)
dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 11534.00 🟢 ( previous job: 5874.00, improvement: 196.36%)
dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 4952.00 🟢 ( previous job: 29.00, improvement: 17075.86%)
dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 307951.00 🟢 ( previous job: 292489.00, improvement: 105.29%)
dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 122060.00 🟢 ( previous job: 110817.00, improvement: 110.15%)
dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 444500.00 🟢 ( previous job: 137802.00, improvement: 322.56%)
dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 203859.00 🟢 ( previous job: 121719.00, improvement: 167.48%)
dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 107596.00 🟢 ( previous job: 103932.00, improvement: 103.53%)
dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 6679.00 🟢 ( previous job: 6356.00, improvement: 105.08%)
dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 8162.00 🟢 ( previous job: 7695.00, improvement: 106.07%)
dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 4701.00 🟢 ( previous job: 3925.00, improvement: 119.77%)
fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 345608.00 🔻 ( previous job: 366891.00, degradation: 94.20%)
fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 257319.00 🟢 ( previous job: 140215.00, improvement: 183.52%)
fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 309588.00 🟢 ( previous job: 299764.00, improvement: 103.28%)
fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 60139.00 🟢 ( previous job: 47575.00, improvement: 126.41%)
fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 81684.00 🔻 ( previous job: 86001.00, degradation: 94.98%)
fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 9349.00 🟢 ( previous job: 9042.00, improvement: 103.40%)
fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 359593.00 🔻 ( previous job: 387500.00, degradation: 92.80%)
fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 241218.00 🟢 ( previous job: 136640.00, improvement: 176.54%)
fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 366122.00 🟢 ( previous job: 325139.00, improvement: 112.60%)
fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 99927.00 🟢 ( previous job: 79539.00, improvement: 125.63%)
fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 82344.00 🔻 ( previous job: 87396.00, degradation: 94.22%)
fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 9310.00 🟢 ( previous job: 8992.00, improvement: 103.54%)
fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 374625.00 🔻 ( previous job: 383531.00, degradation: 97.68%)
fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 298909.00 🟢 ( previous job: 293225.00, improvement: 101.94%)
fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 81325.00 🔻 ( previous job: 87141.00, degradation: 93.33%)
fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8717.00 🔻 ( previous job: 8804.00, degradation: 99.01%)
debian-13-xfce_vm-dispvm (mean:6.515): 78.18
debian-13-xfce_vm-dispvm-gui (mean:7.555): 90.66
debian-13-xfce_vm-dispvm-concurrent (mean:3.009): 36.10
debian-13-xfce_vm-dispvm-gui-concurrent (mean:4.258): 51.09
debian-13-xfce_dom0-dispvm (mean:7.005): 84.06
debian-13-xfce_dom0-dispvm-gui (mean:8.001): 96.01
debian-13-xfce_dom0-dispvm-concurrent (mean:3.46): 41.52
debian-13-xfce_dom0-dispvm-gui-concurrent (mean:4.215): 50.59
debian-13-xfce_vm-dispvm-preload (mean:2.793): 33.52
debian-13-xfce_vm-dispvm-preload-gui (mean:3.88): 46.56
debian-13-xfce_vm-dispvm-preload-concurrent (mean:2.546): 30.55
debian-13-xfce_vm-dispvm-preload-gui-concurrent (mean:3.603): 43.23
debian-13-xfce_dom0-dispvm-preload (mean:3.457): 41.48
debian-13-xfce_dom0-dispvm-preload-gui (mean:10.858): 130.30
debian-13-xfce_dom0-dispvm-preload-concurrent (mean:2.735): 32.82
debian-13-xfce_dom0-dispvm-preload-gui-concurrent (mean:3.655): 43.86
debian-13-xfce_dom0-dispvm-api (mean:6.966): 83.59
debian-13-xfce_dom0-dispvm-gui-api (mean:8.14): 97.68
debian-13-xfce_dom0-dispvm-concurrent-api (mean:3.315): 39.77
debian-13-xfce_dom0-dispvm-gui-concurrent-api (mean:4.003): 48.03
debian-13-xfce_dom0-dispvm-preload-less-less-api (mean:3.794): 45.53
debian-13-xfce_dom0-dispvm-preload-less-api (mean:3.704): 44.45
debian-13-xfce_dom0-dispvm-preload-api (mean:3.564): 42.77
debian-13-xfce_dom0-dispvm-preload-more-api (mean:3.188): 38.26
debian-13-xfce_dom0-dispvm-preload-more-more-api (mean:3.539): 42.47
debian-13-xfce_dom0-dispvm-preload-gui-api (mean:4.306): 51.67
debian-13-xfce_dom0-dispvm-preload-concurrent-api (mean:2.952): 35.42
debian-13-xfce_dom0-dispvm-preload-gui-concurrent-api (mean:3.682): 44.19
debian-13-xfce_vm-vm (mean:0.028): 0.34
debian-13-xfce_vm-vm-gui (mean:0.04): 0.48
debian-13-xfce_vm-vm-concurrent (mean:0.011): 0.13
debian-13-xfce_vm-vm-gui-concurrent (mean:0.021): 0.26
debian-13-xfce_dom0-vm-api (mean:0.034): 0.41
debian-13-xfce_dom0-vm-gui-api (mean:0.038): 0.46
debian-13-xfce_dom0-vm-concurrent-api (mean:0.029): 0.35
debian-13-xfce_dom0-vm-gui-concurrent-api (mean:0.03): 0.36
fedora-42-xfce_vm-dispvm (mean:7.018): 84.22
fedora-42-xfce_vm-dispvm-gui (mean:7.996): 95.95
fedora-42-xfce_vm-dispvm-concurrent (mean:3.727): 44.73
fedora-42-xfce_vm-dispvm-gui-concurrent (mean:4.494): 53.93
fedora-42-xfce_dom0-dispvm (mean:7.713): 92.55
fedora-42-xfce_dom0-dispvm-gui (mean:8.994): 107.93
fedora-42-xfce_dom0-dispvm-concurrent (mean:3.726): 44.71
fedora-42-xfce_dom0-dispvm-gui-concurrent (mean:4.595): 55.14
fedora-42-xfce_vm-dispvm-preload (mean:3.098): 37.17
fedora-42-xfce_vm-dispvm-preload-gui (mean:6.589): 79.07
fedora-42-xfce_vm-dispvm-preload-concurrent (mean:2.914): 34.97
fedora-42-xfce_vm-dispvm-preload-gui-concurrent (mean:3.492): 41.91
fedora-42-xfce_dom0-dispvm-preload (mean:3.662): 43.94
fedora-42-xfce_dom0-dispvm-preload-gui (mean:4.871): 58.45
fedora-42-xfce_dom0-dispvm-preload-concurrent (mean:3.415): 40.99
fedora-42-xfce_dom0-dispvm-preload-gui-concurrent (mean:4.069): 48.83
fedora-42-xfce_dom0-dispvm-api (mean:7.362): 88.34
fedora-42-xfce_dom0-dispvm-gui-api (mean:8.732): 104.78
fedora-42-xfce_dom0-dispvm-concurrent-api (mean:3.7): 44.40
fedora-42-xfce_dom0-dispvm-gui-concurrent-api (mean:4.339): 52.07
fedora-42-xfce_dom0-dispvm-preload-less-less-api (mean:4.181): 50.17
fedora-42-xfce_dom0-dispvm-preload-less-api (mean:4.149): 49.79
fedora-42-xfce_dom0-dispvm-preload-api (mean:4.026): 48.31
fedora-42-xfce_dom0-dispvm-preload-more-api (mean:3.647): 43.76
fedora-42-xfce_dom0-dispvm-preload-more-more-api (mean:3.742): 44.91
fedora-42-xfce_dom0-dispvm-preload-gui-api (mean:4.974): 59.69
fedora-42-xfce_dom0-dispvm-preload-concurrent-api (mean:3.454): 41.45
fedora-42-xfce_dom0-dispvm-preload-gui-concurrent-api (mean:3.967): 47.61
fedora-42-xfce_vm-vm (mean:0.025): 0.30
fedora-42-xfce_vm-vm-gui (mean:0.029): 0.35
fedora-42-xfce_vm-vm-concurrent (mean:0.016): 0.19
fedora-42-xfce_vm-vm-gui-concurrent (mean:0.027): 0.33
fedora-42-xfce_dom0-vm-api (mean:0.037): 0.45
fedora-42-xfce_dom0-vm-gui-api (mean:0.054): 0.65
fedora-42-xfce_dom0-vm-concurrent-api (mean:0.029): 0.35
fedora-42-xfce_dom0-vm-gui-concurrent-api (mean:0.025): 0.30
whonix-workstation-18_vm-dispvm (mean:8.938): 107.25
whonix-workstation-18_vm-dispvm-gui (mean:10.079): 120.95
whonix-workstation-18_vm-dispvm-concurrent (mean:5.196): 62.35
whonix-workstation-18_vm-dispvm-gui-concurrent (mean:5.826): 69.92
whonix-workstation-18_dom0-dispvm-gui (mean:10.691): 128.29
whonix-workstation-18_dom0-dispvm-concurrent (mean:5.32): 63.84
whonix-workstation-18_dom0-dispvm-gui-concurrent (mean:5.44): 65.28
whonix-workstation-18_vm-dispvm-preload (mean:4.118): 49.42
whonix-workstation-18_vm-dispvm-preload-gui (mean:11.353): 136.23
whonix-workstation-18_vm-dispvm-preload-concurrent (mean:3.987): 47.84
whonix-workstation-18_vm-dispvm-preload-gui-concurrent (mean:4.875): 58.49
whonix-workstation-18_dom0-dispvm-preload-gui (mean:6.008): 72.10
whonix-workstation-18_dom0-dispvm-preload-concurrent (mean:4.833): 58.00
whonix-workstation-18_dom0-dispvm-preload-gui-concurrent (mean:5.045): 60.54
whonix-workstation-18_dom0-dispvm-api (mean:9.876): 118.51
whonix-workstation-18_dom0-dispvm-gui-api (mean:10.664): 127.96
whonix-workstation-18_dom0-dispvm-concurrent-api (mean:5.124): 61.49
whonix-workstation-18_dom0-dispvm-gui-concurrent-api (mean:5.583): 66.99
whonix-workstation-18_dom0-dispvm-preload-less-less-api (mean:5.534): 66.40
whonix-workstation-18_dom0-dispvm-preload-less-api (mean:5.715): 68.58
whonix-workstation-18_dom0-dispvm-preload-api (mean:5.492): 65.91
whonix-workstation-18_dom0-dispvm-preload-more-api (mean:5.495): 65.94
whonix-workstation-18_dom0-dispvm-preload-more-more-api (mean:5.469): 65.63
whonix-workstation-18_dom0-dispvm-preload-gui-api (mean:5.954): 71.44
whonix-workstation-18_dom0-dispvm-preload-concurrent-api (mean:4.801): 57.61
whonix-workstation-18_dom0-dispvm-preload-gui-concurrent-api (mean:5.252): 63.02
whonix-workstation-18_vm-vm (mean:0.045): 0.54
whonix-workstation-18_vm-vm-gui (mean:0.062): 0.74
whonix-workstation-18_vm-vm-concurrent (mean:0.02): 0.24
whonix-workstation-18_vm-vm-gui-concurrent (mean:0.031): 0.38
whonix-workstation-18_dom0-vm-api (mean:0.039): 0.47
whonix-workstation-18_dom0-vm-gui-api (mean:0.042): 0.51
whonix-workstation-18_dom0-vm-concurrent-api (mean:0.045): 0.54
whonix-workstation-18_dom0-vm-gui-concurrent-api (mean:0.029): 0.35

marmarek · 2025-10-10T14:58:14Z

VmIPv6Networking_debian-13-xfce: test_001_simple_networking_paused_change_purge_old (failure)
AssertionError: 2 != 0 : Ping by IP on netvm=something -> netvm=som...

Didn't help :(

marmarek · 2025-10-27T12:38:13Z

PipelineRetry

This especially is relevant for paused VMs (like preloaded disposables). If backend was restarted while frontend was paused, qubesd will attempt to reconnect it on unpause, which will trigger the race condition.

marmarek added the openqa-pending label Oct 10, 2025

marmarek force-pushed the xennet-reconnect branch from d1c8da5 to fa30238 Compare October 10, 2025 01:49

This was referenced Oct 10, 2025

Added exceptions to qubes udev device export QubesOS/qubes-linux-utils#130

Merged

Add Application search to qube settings QubesOS/qubes-manager#430

Merged

Skip updating disk size when qube is removed QubesOS/qubes-manager#434

Merged

marmarek force-pushed the xennet-reconnect branch from fa30238 to e151ced Compare October 11, 2025 15:04

marmarek force-pushed the xennet-reconnect branch 2 times, most recently from 3968b6c to 9483bfe Compare October 12, 2025 07:40

qubesos-bot mentioned this pull request Oct 14, 2025

tests: log more info on failure QubesOS/qubes-core-admin#736

Draft

marmarek removed the openqa-pending label Oct 15, 2025

marmarek force-pushed the xennet-reconnect branch from 9483bfe to de2472c Compare October 24, 2025 02:04

marmarek added the openqa-pending label Oct 24, 2025

marmarek removed the openqa-pending label Oct 27, 2025

marmarek force-pushed the xennet-reconnect branch 2 times, most recently from 0761190 to d119ef7 Compare October 28, 2025 03:21

marmarek added the openqa-pending label Oct 28, 2025

marmarek force-pushed the xennet-reconnect branch from d119ef7 to b7af870 Compare October 28, 2025 15:23

Apply fix for quick network reconnect after backend destroy

2b1f93a

This especially is relevant for paused VMs (like preloaded disposables). If backend was restarted while frontend was paused, qubesd will attempt to reconnect it on unpause, which will trigger the race condition.

marmarek force-pushed the xennet-reconnect branch from b7af870 to 2b1f93a Compare October 29, 2025 01:52

marmarek removed the openqa-pending label Nov 3, 2025

marmarek merged commit 2b1f93a into QubesOS:main Nov 19, 2025
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Apply fix for quick network reconnect after backend destroy #1193

Apply fix for quick network reconnect after backend destroy #1193

Uh oh!

marmarek commented Oct 10, 2025

Uh oh!

qubesos-bot commented Oct 10, 2025 •

edited

Loading

Uh oh!

marmarek commented Oct 10, 2025

Uh oh!

marmarek commented Oct 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Apply fix for quick network reconnect after backend destroy #1193

Apply fix for quick network reconnect after backend destroy #1193

Uh oh!

Conversation

marmarek commented Oct 10, 2025

Uh oh!

qubesos-bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

OpenQA test summary

New failures, excluding unstable

Failed tests

Fixed failures

Unstable tests

Performance Tests

Performance degradation:

Remaining performance tests:

Uh oh!

marmarek commented Oct 10, 2025

Uh oh!

marmarek commented Oct 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

qubesos-bot commented Oct 10, 2025 •

edited

Loading