As per our discussion in discord an issue report as well to document this (context: https://discord.com/channels/756513972282195969/1168246645922734180/1420771206080102400)
Whenever a pillager spawns, the EventHandler/onEntitySpawn function (with noRaiderEnabled config enabled) when checking for whether a pillager spawned is near a Raider Outpost, it calls for a structure locating function that's quite heavy to run and calls in worldgen. This when paired with a pillager mob spawner means a server can start lagging behind as this is heavy to run (example Spark Profiler log of this happening: https://spark.lucko.me/TvA1fLO3d9).
A less heavy alternative (like POI lookup as you mentioned, I'm not familiar with how it works under the hood and how fast it'd be) could be a good workaround.
Also just for context, this is happening on MC 1.21.1 with NeoForge 21.1.203 and Utilitarian v1.21.1-0.14.0.