`SYSCALL error: EOF detected` while running `./build-docker.sh`

[thisistheplace]

Hi @StefanHaselwimmer,

This looks fantastic!

I wrote a yaml interfaced python workflow built on top of QGIS to do wind and solar constraints mapping as part of my work on Sharenergy's Community Heat Development Unit project, although not as slick a workflow as you've developed here. I can imagine some of the pain you must have gone through to get this working!

I've had a quick go at running the docker workflow. I'm on an M2 macbook air, 16GB RAM. Docker desktop has access to 12GB of RAM, 2GB swap. I'm getting the following error while processing the public footpaths UK dataset. I assume this is related to memory exceedance. Can you confirm?

Run command:
./build-docker.sh

Console output:

2025-04-16 08:44:38 [INFO] Adding 136.6m buffer: public_footpaths__uk -> public_footpaths__uk__buf_136_6m
2025-04-16 08:46:08 [INFO] Processing table: public_footpaths__uk__buf_136_6m
2025-04-16 08:46:08 [INFO]  --> Step 1: Select only polygons, dump and make valid
2025-04-16 08:46:59 [INFO]  --> Step 2: Clipping partially overlapping polygons
2025-04-16 09:44:01 [INFO]  --> Step 3: Adding fully enclosed polygons
2025-04-16 09:50:02 [INFO]  --> Step 4: Dumping and removing non-polygons
Traceback (most recent call last):
  File "/usr/src/openwind/openwind.py", line 3064, in <module>
    runProcessingOnDownloads(DATASETS_DOWNLOADS_FOLDER)
  File "/usr/src/openwind/openwind.py", line 2398, in runProcessingOnDownloads
    postgisExec("CREATE TABLE %s AS SELECT dumped.geom FROM (SELECT (ST_Dump(ST_Union(geom))).geom geom FROM %s) AS dumped WHERE ST_geometrytype(dumped.geom) = 'ST_Polygon';", (AsIs(processed_table), AsIs(scratch_table_2), ))
  File "/usr/src/openwind/openwind.py", line 349, in postgisExec
    cur.execute(sql_text, sql_parameters)
psycopg2.OperationalError: SSL SYSCALL error: EOF detected

I'm getting a couple of other errors when attempting to use small configurations, but I'll raise these as separate issues.

Thanks!

Ben

[open-wind]

Hi Ben,

Yes, we've seen this before and it's memory-related - can you perhaps nudge up to 13Gb? Also we'll have a think about how we can maybe break the PostGIS queries down into separate chunks to mitigate this.

Meanwhile, two workarounds:

(i) Try and set up a non-Docker install. We've successfully run a non-Docker build on a Macbook Air 2015 with 8Gb of RAM
(ii) Try the (soon-to-be-tested-and-finalised) '--custom' option, for example './build-cli.sh --custom england'

Let us know how you get on!

Best,

Stefan