Skip to content

Conversation

@bedroge
Copy link
Contributor

@bedroge bedroge commented Dec 16, 2025

See EESSI/software-layer#1341, many Arm builds failed because they ran out of memory.

@bedroge
Copy link
Contributor Author

bedroge commented Dec 16, 2025

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1

@eessi-bot-aws
Copy link

eessi-bot-aws bot commented Dec 16, 2025

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: neoverse_v1
Building for: aarch64/neoverse_v1
Job dir: /project/def-users/SHARED/jobs/2025.12/pr_139/113513

date job status comment
Dec 16 20:58:43 UTC 2025 submitted job id 113513 awaits release by job manager
Dec 16 20:59:02 UTC 2025 released job awaits launch by Slurm scheduler
Dec 16 21:04:11 UTC 2025 running job 113513 is running
Dec 17 00:27:50 UTC 2025 finished
😢 FAILURE (click triangle for details)
Details
✅ job output file slurm-113513.out
✅ no message matching FATAL:
❌ found message matching ERROR:
❌ found message matching FAILED:
❌ found message matching required modules missing:
❌ no message matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2025.06-software-linux-aarch64-neoverse_v1-17659297050.tar.zstsize: 138 MiB (145657630 bytes)
entries: 5595
modules under 2025.06/software/linux/aarch64/neoverse_v1/modules/all
assimp/6.0.2-GCCcore-14.2.0.lua
double-conversion/3.3.1-GCCcore-14.2.0.lua
FFmpeg/7.1.1-GCCcore-14.2.0.lua
ffnvcodec/13.0.19.0.lua
Gdk-Pixbuf/2.42.12-GCCcore-14.2.0.lua
graphite2/1.3.14-GCCcore-14.2.0.lua
JasPer/4.2.5-GCCcore-14.2.0.lua
LAME/3.100-GCCcore-14.2.0.lua
libde265/1.0.16-GCCcore-14.2.0.lua
libheif/1.19.8-GCCcore-14.2.0.lua
nodejs/22.16.0-GCCcore-14.2.0.lua
NSPR/4.36-GCCcore-14.2.0.lua
NSS/3.113-GCCcore-14.2.0.lua
re2c/4.2-GCCcore-14.2.0.lua
SDL2/2.32.8-GCCcore-14.2.0.lua
snappy/1.2.2-GCCcore-14.2.0.lua
x264/20250619-GCCcore-14.2.0.lua
x265/4.1-GCCcore-14.2.0.lua
software under 2025.06/software/linux/aarch64/neoverse_v1/software
assimp/6.0.2-GCCcore-14.2.0
double-conversion/3.3.1-GCCcore-14.2.0
FFmpeg/7.1.1-GCCcore-14.2.0
ffnvcodec/13.0.19.0
Gdk-Pixbuf/2.42.12-GCCcore-14.2.0
graphite2/1.3.14-GCCcore-14.2.0
JasPer/4.2.5-GCCcore-14.2.0
LAME/3.100-GCCcore-14.2.0
libde265/1.0.16-GCCcore-14.2.0
libheif/1.19.8-GCCcore-14.2.0
nodejs/22.16.0-GCCcore-14.2.0
NSPR/4.36-GCCcore-14.2.0
NSS/3.113-GCCcore-14.2.0
re2c/4.2-GCCcore-14.2.0
SDL2/2.32.8-GCCcore-14.2.0
snappy/1.2.2-GCCcore-14.2.0
x264/20250619-GCCcore-14.2.0
x265/4.1-GCCcore-14.2.0
reprod directories under 2025.06/software/linux/aarch64/neoverse_v1/reprod
assimp/6.0.2-GCCcore-14.2.0/20251216_210802UTC
double-conversion/3.3.1-GCCcore-14.2.0/20251216_210609UTC
FFmpeg/7.1.1-GCCcore-14.2.0/20251216_221146UTC
ffnvcodec/13.0.19.0/20251216_211003UTC
Gdk-Pixbuf/2.42.12-GCCcore-14.2.0/20251216_211001UTC
graphite2/1.3.14-GCCcore-14.2.0/20251216_210627UTC
JasPer/4.2.5-GCCcore-14.2.0/20251216_211258UTC
LAME/3.100-GCCcore-14.2.0/20251216_220621UTC
libde265/1.0.16-GCCcore-14.2.0/20251216_210823UTC
libheif/1.19.8-GCCcore-14.2.0/20251216_211202UTC
nodejs/22.16.0-GCCcore-14.2.0/20251216_220550UTC
NSPR/4.36-GCCcore-14.2.0/20251216_211433UTC
NSS/3.113-GCCcore-14.2.0/20251216_212633UTC
re2c/4.2-GCCcore-14.2.0/20251216_210547UTC
SDL2/2.32.8-GCCcore-14.2.0/20251216_220746UTC
snappy/1.2.2-GCCcore-14.2.0/20251216_211341UTC
x264/20250619-GCCcore-14.2.0/20251216_211404UTC
x265/4.1-GCCcore-14.2.0/20251216_210919UTC
other under 2025.06/software/linux/aarch64/neoverse_v1
2025.06/init/easybuild/eb_hooks.py
Dec 17 00:27:50 UTC 2025 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] (1/4) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /e4bf9965 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 1.57 us (r:0, l:None, u:None)
[ OK ] (2/4) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /3da4890b @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 4.78 us (r:0, l:None, u:None)
[ OK ] (3/4) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /3255009a @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 0.26 us (r:0, l:None, u:None)
[ OK ] (4/4) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /59f4b331 @BotBuildTests:aarch64_neoverse_v1+default
P: bandwidth: 28960.78 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-113513.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@bedroge
Copy link
Contributor Author

bedroge commented Dec 17, 2025

Still getting these errors:

virtual memory exhausted: Cannot allocate memory
as: out of memory allocating 4064 bytes after a total of 11206656 bytes

So let's try with even fewer cores...

@bedroge
Copy link
Contributor Author

bedroge commented Dec 17, 2025

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1

@eessi-bot-aws
Copy link

eessi-bot-aws bot commented Dec 17, 2025

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: neoverse_v1
Building for: aarch64/neoverse_v1
Job dir: /project/def-users/SHARED/jobs/2025.12/pr_139/113787

date job status comment
Dec 17 06:27:48 UTC 2025 submitted job id 113787 awaits release by job manager
Dec 17 06:28:23 UTC 2025 released job awaits launch by Slurm scheduler
Dec 17 06:33:26 UTC 2025 running job 113787 is running
Dec 17 10:16:52 UTC 2025 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-113787.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2025.06-software-linux-aarch64-neoverse_v1-17659665030.tar.zstsize: 378 MiB (397285375 bytes)
entries: 24687
modules under 2025.06/software/linux/aarch64/neoverse_v1/modules/all
assimp/6.0.2-GCCcore-14.2.0.lua
double-conversion/3.3.1-GCCcore-14.2.0.lua
FFmpeg/7.1.1-GCCcore-14.2.0.lua
ffnvcodec/13.0.19.0.lua
Gdk-Pixbuf/2.42.12-GCCcore-14.2.0.lua
graphite2/1.3.14-GCCcore-14.2.0.lua
JasPer/4.2.5-GCCcore-14.2.0.lua
LAME/3.100-GCCcore-14.2.0.lua
libde265/1.0.16-GCCcore-14.2.0.lua
libheif/1.19.8-GCCcore-14.2.0.lua
nodejs/22.16.0-GCCcore-14.2.0.lua
NSPR/4.36-GCCcore-14.2.0.lua
NSS/3.113-GCCcore-14.2.0.lua
Qt6/6.9.3-GCCcore-14.2.0.lua
re2c/4.2-GCCcore-14.2.0.lua
SDL2/2.32.8-GCCcore-14.2.0.lua
snappy/1.2.2-GCCcore-14.2.0.lua
x264/20250619-GCCcore-14.2.0.lua
x265/4.1-GCCcore-14.2.0.lua
software under 2025.06/software/linux/aarch64/neoverse_v1/software
assimp/6.0.2-GCCcore-14.2.0
double-conversion/3.3.1-GCCcore-14.2.0
FFmpeg/7.1.1-GCCcore-14.2.0
ffnvcodec/13.0.19.0
Gdk-Pixbuf/2.42.12-GCCcore-14.2.0
graphite2/1.3.14-GCCcore-14.2.0
JasPer/4.2.5-GCCcore-14.2.0
LAME/3.100-GCCcore-14.2.0
libde265/1.0.16-GCCcore-14.2.0
libheif/1.19.8-GCCcore-14.2.0
nodejs/22.16.0-GCCcore-14.2.0
NSPR/4.36-GCCcore-14.2.0
NSS/3.113-GCCcore-14.2.0
Qt6/6.9.3-GCCcore-14.2.0
re2c/4.2-GCCcore-14.2.0
SDL2/2.32.8-GCCcore-14.2.0
snappy/1.2.2-GCCcore-14.2.0
x264/20250619-GCCcore-14.2.0
x265/4.1-GCCcore-14.2.0
reprod directories under 2025.06/software/linux/aarch64/neoverse_v1/reprod
assimp/6.0.2-GCCcore-14.2.0/20251217_063703UTC
double-conversion/3.3.1-GCCcore-14.2.0/20251217_063510UTC
FFmpeg/7.1.1-GCCcore-14.2.0/20251217_074102UTC
ffnvcodec/13.0.19.0/20251217_063904UTC
Gdk-Pixbuf/2.42.12-GCCcore-14.2.0/20251217_063901UTC
graphite2/1.3.14-GCCcore-14.2.0/20251217_063528UTC
JasPer/4.2.5-GCCcore-14.2.0/20251217_064159UTC
LAME/3.100-GCCcore-14.2.0/20251217_073537UTC
libde265/1.0.16-GCCcore-14.2.0/20251217_063723UTC
libheif/1.19.8-GCCcore-14.2.0/20251217_064103UTC
nodejs/22.16.0-GCCcore-14.2.0/20251217_073506UTC
NSPR/4.36-GCCcore-14.2.0/20251217_064334UTC
NSS/3.113-GCCcore-14.2.0/20251217_065540UTC
Qt6/6.9.3-GCCcore-14.2.0/20251217_101435UTC
re2c/4.2-GCCcore-14.2.0/20251217_063448UTC
SDL2/2.32.8-GCCcore-14.2.0/20251217_073702UTC
snappy/1.2.2-GCCcore-14.2.0/20251217_064242UTC
x264/20250619-GCCcore-14.2.0/20251217_064305UTC
x265/4.1-GCCcore-14.2.0/20251217_063819UTC
other under 2025.06/software/linux/aarch64/neoverse_v1
2025.06/init/easybuild/eb_hooks.py
Dec 17 10:16:52 UTC 2025 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] (1/4) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /e4bf9965 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 1.65 us (r:0, l:None, u:None)
[ OK ] (2/4) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /3da4890b @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 4.78 us (r:0, l:None, u:None)
[ OK ] (3/4) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /3255009a @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 0.24 us (r:0, l:None, u:None)
[ OK ] (4/4) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /59f4b331 @BotBuildTests:aarch64_neoverse_v1+default
P: bandwidth: 28530.4 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-113787.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@bedroge
Copy link
Contributor Author

bedroge commented Dec 17, 2025

That worked 🎉 Removed the easystack, let's do the final build.

bot: build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=aarch64/neoverse_v1

@eessi-bot-aws
Copy link

eessi-bot-aws bot commented Dec 17, 2025

New job on instance eessi-bot-mc-aws for repository eessi.io-2023.06-software
Building on: neoverse_v1
Building for: aarch64/neoverse_v1
Job dir: /project/def-users/SHARED/jobs/2025.12/pr_139/113808

date job status comment
Dec 17 10:31:55 UTC 2025 submitted job id 113808 awaits release by job manager
Dec 17 10:32:34 UTC 2025 released job awaits launch by Slurm scheduler
Dec 17 10:33:43 UTC 2025 running job 113808 is running
Dec 17 10:37:00 UTC 2025 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-113808.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2023.06-software-linux-aarch64-neoverse_v1-17659675840.tar.zstsize: 0 MiB (22874 bytes)
entries: 1
modules under 2023.06/software/linux/aarch64/neoverse_v1/modules/all
no module files in tarball
software under 2023.06/software/linux/aarch64/neoverse_v1/software
no software packages in tarball
reprod directories under 2023.06/software/linux/aarch64/neoverse_v1/reprod
no reprod directories in tarball
other under 2023.06/software/linux/aarch64/neoverse_v1
2023.06/init/easybuild/eb_hooks.py
Dec 17 10:37:00 UTC 2025 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:aarch64_neoverse_v1+default
P: perf: 944.066 timesteps/s (r:0, l:None, u:None)
[ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:aarch64_neoverse_v1+default
P: perf: 972.738 timesteps/s (r:0, l:None, u:None)
[ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 3.24 us (r:0, l:None, u:None)
[ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 3.02 us (r:0, l:None, u:None)
[ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 4.27 us (r:0, l:None, u:None)
[ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 5.57 us (r:0, l:None, u:None)
[ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 0.48 us (r:0, l:None, u:None)
[ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 0.4 us (r:0, l:None, u:None)
[ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:aarch64_neoverse_v1+default
P: bandwidth: 30611.48 MB/s (r:0, l:None, u:None)
[ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:aarch64_neoverse_v1+default
P: bandwidth: 21626.44 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-113808.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case
Dec 17 10:43:49 UTC 2025 uploaded transfer of eessi-2023.06-software-linux-aarch64-neoverse_v1-17659675840.tar.zst to S3 bucket succeeded

@eessi-bot-aws
Copy link

eessi-bot-aws bot commented Dec 17, 2025

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: neoverse_v1
Building for: aarch64/neoverse_v1
Job dir: /project/def-users/SHARED/jobs/2025.12/pr_139/113809

date job status comment
Dec 17 10:32:00 UTC 2025 submitted job id 113809 awaits release by job manager
Dec 17 10:32:31 UTC 2025 released job awaits launch by Slurm scheduler
Dec 17 10:33:44 UTC 2025 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-113809.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2025.06-software-linux-aarch64-neoverse_v1-17659675660.tar.zstsize: 0 MiB (22879 bytes)
entries: 1
modules under 2025.06/software/linux/aarch64/neoverse_v1/modules/all
no module files in tarball
software under 2025.06/software/linux/aarch64/neoverse_v1/software
no software packages in tarball
reprod directories under 2025.06/software/linux/aarch64/neoverse_v1/reprod
no reprod directories in tarball
other under 2025.06/software/linux/aarch64/neoverse_v1
2025.06/init/easybuild/eb_hooks.py
Dec 17 10:33:44 UTC 2025 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] (1/4) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /e4bf9965 @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 1.58 us (r:0, l:None, u:None)
[ OK ] (2/4) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /3da4890b @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 4.69 us (r:0, l:None, u:None)
[ OK ] (3/4) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /3255009a @BotBuildTests:aarch64_neoverse_v1+default
P: latency: 0.26 us (r:0, l:None, u:None)
[ OK ] (4/4) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /59f4b331 @BotBuildTests:aarch64_neoverse_v1+default
P: bandwidth: 29129.06 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 4/4 test case(s) from 4 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-113809.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case
Dec 17 10:43:41 UTC 2025 uploaded transfer of eessi-2025.06-software-linux-aarch64-neoverse_v1-17659675660.tar.zst to S3 bucket succeeded

Copy link
Member

@ocaisa ocaisa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ocaisa ocaisa merged commit d5a9bb8 into EESSI:main Dec 17, 2025
66 of 68 checks passed
@bedroge bedroge deleted the qt6_limits branch December 17, 2025 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants