improve fixtures for mss_dir and eventlet server #2959

ReimarBauer · 2025-12-17T08:54:04Z

Purpose of PR?:

tests/fixtures.py

matrss · 2025-12-22T08:27:28Z

tests/fixtures.py

        handle_db_reset()


+@pytest.fixture(scope="session")


Shouldn't this temporary directory be newly created for every test to avoid implicit dependencies?

oh, yes, good catch

matrss · 2025-12-22T10:27:59Z

Are you sure the changes in d67169f are necessary? CI passes without them on macOS runners with the usual (un-)reliability.

matrss · 2025-12-22T10:14:09Z

tests/fixtures.py

    ctx = multiprocessing.get_context("fork")
-    process = ctx.Process(target=eventlet.wsgi.server, args=(socket, app), daemon=True)
+
+    def _serve_in_child(conn, _app, _host, _scheme):


Starting parameters with an underscore usually indicates that they are unused AFAIU.

matrss · 2025-12-22T10:15:04Z

tests/fixtures.py

+        try:
+            _app.config["URL"] = url
+        except Exception:
+            # If app/config is not writable for some reason, still start server.
+            pass


This is bad because now you can't rely on the correctness of this config value. If it is used somewhere then it must be set properly, if it is not used then we should never set it, IMO.

ReimarBauer · 2025-12-22T10:57:12Z

Are you sure the changes in d67169f are necessary? CI passes without them on macOS runners with the usual (un-)reliability.

I recheck, I had first tried a PR with the first none optimal change. Lets see.

ReimarBauer · 2025-12-22T12:11:23Z

on my macOS 26.2 system I get currently

many "RuntimeError: Server did not start within 5 seconds"

18 passed, 16 skipped, 67 warnings, 358 errors in 84.93s (0:01:24)
I will retry after a fresh reboot.

ReimarBauer · 2025-12-22T13:28:29Z

=============================================================== short test summary info ===============================================================
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_server_service_cache - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_multilayer_handling - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_filter_handling - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_multilayer_syncing - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_server_getmap - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_invalid_url - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_no_server - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_invalid_schema - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_singlelayer_handling - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52487
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_no_schema - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52487
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_server_getmap_cached - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52487
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_server_no_thread - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52487
ERROR tests/_test_msui/test_sideview.py::Test_SideViewWMS::test_server_getmap - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:52141
========================================= 563 passed, 16 skipped, 92 warnings, 13 errors in 110.51s (0:01:50) =========================================

Here we see an attempt to use 8 times 52141 and 4 times 52487

ReimarBauer · 2025-12-22T13:38:53Z

Another run 567 passed, 16 skipped, 92 warnings, 9 errors in 113.76s (0:01:53)

ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_invalid_url - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:54919
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_no_server - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:54919
ERROR tests/_test_msui/test_linearview.py::Test_LinearViewWMS::test_server_getmap - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:54977
ERROR tests/_test_msui/test_wms_control.py::Test_VSecWMSControlWidget::test_server_getmap - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:55162
ERROR tests/_test_msui/test_wms_control.py::Test_VSecWMSControlWidget::test_multilayer_drawing - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:55162
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_invalid_schema - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:55162
ERROR tests/_test_msui/test_wms_control.py::Test_HSecWMSControlWidget::test_filter_handling - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:55162
ERROR tests/_test_msui/test_sideview.py::Test_SideViewWMS::test_server_getmap - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:55162
ERROR tests/_test_msui/test_topview.py::Test_TopViewWMS::test_server_getmap - RuntimeError: Server did not start within 5 seconds at http://127.0.0.1:55359

Maybe the problem is that not each test has a different port? here we see 2 times 54919, 5 times 55162

matrss · 2025-12-22T14:14:18Z

Port selection is not the issue, the OS selects a free port for us which should be race-free even in the presence of many unrelated processes. The reason why multiple errors show the same port number is because those tests share the same fixture instance for the server. E.g. a mswms server fixture is requested at the class-level here:

MSS/tests/_test_msui/test_wms_control.py

Lines 52 to 56 in 27a2fb1

    
           @pytest.fixture(autouse=True) 
        
           def _with_mswms_server(self, mswms_server): 
        
               self.url = mswms_server 
        
               parsed_url = urllib.parse.urlparse(self.url) 
        
               self.scheme, self.host, self.port = parsed_url.scheme, parsed_url.hostname, parsed_url.port

. This server instance is shared between all tests in that class, or rather its subclasses (but not /between/ subclasses, it seems).

matrss · 2025-12-22T14:16:45Z

FWIW I know that it passes in CI because I tried it here: https://github.com/matrss/MSS/actions/runs/20426406249/job/58694840047

It just required a bunch of restarts of the macOS 14 tests, which seem to be the most unreliable. But I am not (yet) convinced that the additional changes made it any better.

ReimarBauer · 2025-12-23T11:30:18Z

FWIW I know that it passes in CI because I tried it here: https://github.com/matrss/MSS/actions/runs/20426406249/job/58694840047

It just required a bunch of restarts of the macOS 14 tests, which seem to be the most unreliable. But I am not (yet) convinced that the additional changes made it any better.

On github there is usually not a ~/mss for the account running the tests. But for a user trying the mswms with demodata has some python config files there.
A user trying mscolab does have there also a colabdata.

Tests should not have the option to change user data or can get different results because of userdata.

matrss · 2025-12-23T18:45:34Z

I meant the tests pass in CI without the server startup modifications but with the change to the mss_dir configuration, so I don't quite understand your comment...

ReimarBauer added 2 commits December 17, 2025 09:42

mss_dir in tests as tmp_path

18a9d8a

use a child for the listening socket

d67169f

ReimarBauer commented Dec 17, 2025

View reviewed changes

tests/fixtures.py Outdated Show resolved Hide resolved

ReimarBauer commented Dec 17, 2025

View reviewed changes

tests/fixtures.py Outdated Show resolved Hide resolved

ReimarBauer requested a review from matrss December 17, 2025 09:35

ReimarBauer linked an issue Dec 17, 2025 that may be closed by this pull request

we need to change the fixture to have a tmp_path for mss_dir #2958

Open

ReimarBauer requested a review from joernu76 December 19, 2025 16:12

matrss reviewed Dec 22, 2025

View reviewed changes

ReimarBauer added 3 commits December 22, 2025 12:46

set tmp_path only

10373ac

undo changes

449b9fe

linter

7475d56

ReimarBauer requested a review from matrss December 22, 2025 13:42

		handle_db_reset()


		@pytest.fixture(scope="session")

improve fixtures for mss_dir and eventlet server #2959

Are you sure you want to change the base?

improve fixtures for mss_dir and eventlet server #2959

Uh oh!

Conversation

ReimarBauer commented Dec 17, 2025

Uh oh!

Uh oh!

Uh oh!

matrss Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

ReimarBauer Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

matrss commented Dec 22, 2025

Uh oh!

matrss Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

matrss Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

ReimarBauer commented Dec 22, 2025

Uh oh!

ReimarBauer commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ReimarBauer commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ReimarBauer commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matrss commented Dec 22, 2025

Uh oh!

matrss commented Dec 22, 2025

Uh oh!

ReimarBauer commented Dec 23, 2025

Uh oh!

matrss commented Dec 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ReimarBauer commented Dec 22, 2025 •

edited

Loading

ReimarBauer commented Dec 22, 2025 •

edited

Loading

ReimarBauer commented Dec 22, 2025 •

edited

Loading