docs: document perlmutter gasnet requirements (#3073)

Author: marcinz

conda/conda-build/meta.yaml

@@ -186,7 +186,7 @@ requirements:
 {% if gpu_enabled_bool %}
     - cuda-version ={{ cuda_version }}
     # these are all constrained by cuda-version
-    - nccl
+    - nccl           >=2.0,<2.28
     - cuda-cudart-dev
     - cuda-nvtx-dev
     - cuda-nvml-dev
@@ -218,7 +218,7 @@ requirements:
     - {{ pin_compatible('cuda-cudart',  min_pin='x', max_pin='x') }}
     - {{ pin_compatible('cuda-nvtx',    min_pin='x', max_pin='x') }}
     - libcufile      >=1.0,<2
-    - nccl           >=2.0
+    - nccl           >=2.0,<2.28
 {% endif %}
     - rich
 {% if network == 'ucx' %}

continuous_integration/requirements-build.txt

@@ -2,7 +2,7 @@
 --extra-index-url=https://pypi.nvidia.com
 cmake>=3.26.4,!=3.30.0
 ninja
-nvidia-nccl-cu12<=2.27
+nvidia-nccl-cu12<2.28
 libucx-cu12
 nvidia-libcal-cu12
 scikit-build-core[pyproject]>=0.10.0

docs/legate/source/gasnet.rst

@@ -201,6 +201,13 @@ install cmake`` or by any other means.  Note that when the wrapper is built, the
 final message suggests reactivating the environment, but that is not necessary
 before building the GASNet wrapper:
 
+.. note::
+
+   As of October 2025, the GASNet wrapper on Perlmutter only works when the
+   NERSC-provided ``mpich`` module is loaded.  Attempts to build or use the
+   wrapper with ``cray-mpich`` currently fail, so ensure ``module load mpich``
+   is issued before running ``build-gex-wrapper.sh``.
+
 .. code-block:: sh
 
     login40:~> /conda/envs/legate-gex-anaconda/gex-wrapper/build-gex-wrapper.sh
@@ -339,6 +346,14 @@ actually provide the ``srun`` command around Legate, and then we would use
 launcher.  With our options, Legate run results in the following output on
 Perlmutter:
 
+.. warning::
+
+   As of October 2025, Perlmutter jobs that request more than 32 GB of frame
+   buffer memory (for example, ``--fbmem 64000``) must include ``-gex:bindcuda
+   0`` in the options passed through ``REALM_DEFAULT_ARGS``.  Otherwise the OFI
+   provider aborts with ``Unexpected error 12 (Cannot allocate memory) from
+   fi_mr_regattr()``.
+
 .. code-block:: sh
 
     --------------------- CONDA/MPI_WRAPPER/ACTIVATE.SH -----------------------