Rewrite test suite to pytest & run CTest

[ekohl]

The old test suite was using the console connection but in 7af2df3 this was removed.

Testing is useful so this introduces a basic test suite using pytest because that's what I'm familiar with.

[ekohl]

Somehow this sometimes hangs and I'm not sure why yet.

[teddych]

What do you mean with your last post?

[ekohl]

You can see that ubuntu-latest with make was cancelled after 360m. I'm not sure why that happens, but I have also seen it locally. Perhaps some sort of race condition since I am polling until the service is up, though I have a maximum count and timeout.

If you have thoughts then I'd welcome them.

[teddych]

I can reproduce it too. I expect the problem by starting railcontrol from another directory than ".". Because then all static files of the webserver are not served anymore.

[ekohl]

I can reproduce it too.

The odd thing is that if I locally run the same command again it does pass. Any idea why that could be?

I expect the problem by starting railcontrol from another directory than ".". Because then all static files of the webserver are not served anymore.

That's also my theory and #22 is probably part of a bigger issue: you can't run railcontrol from an arbitrary path. Solving that would make testing easier.

Perhaps a short term fix could be to find the right options so that each test uses its own database in the same build directory and not write out any log file, relying on stdout. Hopefully that provides enough test isolation.

[ekohl]

I'm going to further experiment in a different branch because here I can't cancel CI.

[ekohl]

Output from a run with more debug output gives me the impression the problem 2 fold:

• The tests poll too fast (no waiting in between attempts)
• Raising an exception fixture code doesn't halt and instead results in it being stuck

---------------------------- Captured stdout setup -----------------------------
1: Starting /tmp/pytest-of-runner/pytest-0/test_cmd_getlocolist_no_locos0/railcontrol
1: Started 2902
1: ----------------------------- Captured stdout call -----------------------------
1: Polling 2902
1: Pinging http://localhost:8022
1: 2026-01-21 13:23:18.200098: Info: Main: Starting RailControl
1: 2026-01-21 13:23:18.200159: Info: Main: Version: 24
1: 2026-01-21 13:23:18.200169: Info: Main: Compile date: 2026-01-21 13:22:24
1: 2026-01-21 13:23:18.200174: Info: Main: Last GIT commit hash: 87f3fbf83287c40de301abafd329a4b31e3020e6
1: 2026-01-21 13:23:18.200181: Info: Main: Last GIT commit date: 2026-01-21 13:22:04
1: 2026-01-21 13:23:18.200185: Info: Main: Start argument: /tmp/pytest-of-runner/pytest-0/test_cmd_getlocolist_no_locos0/railcontrol
1: 2026-01-21 13:23:18.200193: Info: Config: Reading config file /tmp/pytest-of-runner/pytest-0/test_cmd_getlocolist_no_locos0/config.conf
1: 2026-01-21 13:23:18.200236: Info: Config: Parameter found in config file: dbfilename = 'railcontrol.sqlite'
1: 2026-01-21 13:23:18.200249: Info: Config: Parameter found in config file: webserverport = 8022
1: 2026-01-21 13:23:18.200254: Info: Config: Parameter found in config file: numkeepbackups = 0
1: 2026-01-21 13:23:18.200297: Info: SQLite: Opening SQLite database with filename 'railcontrol.sqlite'
1: 2026-01-21 13:23:18.200723: Info: SQLite: Creating table hardware
1: Polling 2902
1: Pinging http://localhost:8022
1: Polling 2902
1: Pinging http://localhost:8022
1: Polling 2902
1: Pinging http://localhost:8022
1: Polling 2902
1: Pinging http://localhost:8022
1: 2026-01-21 13:23:18.203835: Info: SQLite: Creating table objects
1: Polling 2902
1: Pinging http://localhost:8022
1: Polling 2902
1: Pinging http://localhost:8022
1: Polling 2902
1: Pinging http://localhost:8022
1: 2026-01-21 13:23:18.206017: Info: SQLite: Creating table relations
1: Polling 2902
1: Pinging http://localhost:8022
1: Polling 2902
1: Pinging http://localhost:8022
1: --------------------------- Captured stdout teardown ---------------------------
1: 2026-01-21 13:23:18.209986: Info: WebServer: Please type one of the following links in your browser to connect to RailControl:
1:    http://localhost:8022/
1:    http://127.0.0.1:8022/
1:    http://10.1.0.24:8022/
1:    http://172.17.0.1:8022/
1:    http://[::1]:8022/
1:    http://[fe80::7eed:8dff:fe09:796e]:8022/

[ekohl]

Raising an exception fixture code doesn't halt and instead results in it being stuck

Reasons for this:

• https://docs.python.org/3/library/subprocess.html#popen-constructor states

Popen objects are supported as context managers via the with statement: on exit, standard file descriptors are closed, and the process is waited for.

And https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#handling-errors-for-yield-fixture states:

If a yield fixture raises an exception before yielding, pytest won’t try to run the teardown code after that yield fixture’s yield statement.

So nothing kills railcontrol but the contextmanager does wait for it. Adding an explicit kill should help.

[ekohl]

Updated. It now includes #33, a commit to limit CI job execution to 15 minutes and 2 fixes:

• Proper pytest fixture setup failure handling
• A sleep between poll attempts

I can see the first results are showing green so I think this is ready.

[ekohl]

Windows now fails with:

C:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/15.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -ldl: No such file or directory
C:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/15.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: have you installed the static version of the dl library ?
collect2.exe: error: ld returned 1 exit status

The initial build passes so I suspect the Python setup somehow breaks the setup.

[teddych]

The test is compiled with another compiler than the build. I guess the other mingw compiler does not find the library. In a standard cygwin the mingw compiler is not installed, I don't know where it comes from.

[ekohl]

I think the setup-python action changed the environment and since the build isn't idempotent (Version.cpp is rewritten every time) it recompiled (since I added railcontrol as a dependency of make's test target.

[teddych]

There are merge conflicts.

[ekohl]

Yes, GH doesn't do well with multiple PRs included. Trivial rebase done.