NotFoundError: undefined symbol: __cudaPushCallConfiguration

[Peter1101Zhang]

When I executed the Python script using the command line, I encountered an error.

python -m python.slalom.scripts.eval vgg_16 sgxdnn --batch_size=8 --max_num_batches=4 --verify --verify_batched --verify_preproc --use_sgx
(tfback) $ python -m python.slalom.scripts.eval vgg_16 sgxdnn --batch_size=8 --max_num_batches=4 --verify --verify_batched --verify_preproc --use_sgx

/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:521: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:522: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Using TensorFlow backend.
Traceback (most recent call last):
File "/miniconda3/envs/tfback/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/miniconda3/envs/tfback/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "code/codetest/slalom-master/python/slalom/scripts/eval.py", line 17, in
from python.slalom.quant_layers import transform
File "/code/codetest/slalom-master/python/slalom/quant_layers.py", line 20, in
fmod_module = tf.load_op_library('./App/cuda_fmod.so')
File "/miniconda3/envs/tfback/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: ./App/cuda_fmod.so: undefined symbol: __cudaPushCallConfiguration

I checked some documents, which indicated that the problem might be due to the incompatibility between the TensorFlow version and the CUDA version. My CUDA version is 9.0, and the TensorFlow version is 1.8.0 GPU. According to the official documentation of TensorFlow, these two should be compatible.Specifically, my operating environment is as follows.

packages in environment at /miniconda3/envs/tfback:

Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
absl-py 1.4.0 pypi_0 pypi
alsa-lib 1.2.3.2 h166bdaf_0 conda-forge
astor 0.8.1 pypi_0 pypi
bleach 1.5.0 pypi_0 pypi
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.18.1 h7f98852_0 conda-forge
ca-certificates 2025.4.26 hbd8a1cb_0 http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
cached-property 1.5.2 pypi_0 pypi
cairo 1.16.0 h6cf1ce9_1008 conda-forge
certifi 2021.5.30 py36h06a4308_0
cudatoolkit 9.0 h13b8566_0
cudnn 7.6.4 cuda9.0_0
dataclasses 0.8 pypi_0 pypi
dbus 1.13.6 h48d8840_2 conda-forge
expat 2.4.8 h27087fc_0 conda-forge
ffmpeg 4.3.2 hca11adc_0 conda-forge
fontconfig 2.14.0 h8e229c2_0 conda-forge
freetype 2.10.4 h0708190_1 conda-forge
gast 0.6.0 pypi_0 pypi
gettext 0.19.8.1 h0b5b191_1005 conda-forge
glib 2.68.4 h9c3ff4c_0 conda-forge
glib-tools 2.68.4 h9c3ff4c_0 conda-forge
gmp 6.2.1 h58526e2_0 conda-forge
gnutls 3.6.13 h85f3911_1 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
grpcio 1.48.2 pypi_0 pypi
gst-plugins-base 1.18.5 hf529b03_0 conda-forge
gstreamer 1.18.5 h76c114f_0 conda-forge
h5py 3.1.0 pypi_0 pypi
harfbuzz 2.9.1 h83ec7ef_1 conda-forge
hdf5 1.12.1 nompi_h2750804_100 conda-forge
html5lib 0.9999999 pypi_0 pypi
icu 68.2 h9c3ff4c_0 conda-forge
importlib-metadata 4.8.3 pypi_0 pypi
jasper 1.900.1 h07fcdf6_1006 conda-forge
jbig 2.1 h7f98852_2003 conda-forge
jpeg 9e h166bdaf_1 conda-forge
keras 2.1.6 pypi_0 pypi
keyutils 1.6.1 h166bdaf_0 conda-forge
krb5 1.19.3 h3790be6_0 conda-forge
lame 3.100 h7f98852_1001 conda-forge
ld_impl_linux-64 2.40 h12ee557_0
lerc 2.2.1 h9c3ff4c_0 conda-forge
libblas 3.9.0 16_linux64_openblas conda-forge
libcblas 3.9.0 16_linux64_openblas conda-forge
libclang 11.1.0 default_ha53f305_1 conda-forge
libcurl 7.79.1 h2574ce0_1 conda-forge
libdeflate 1.7 h7f98852_5 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libevent 2.1.10 h9b69904_4 conda-forge
libffi 3.3 he6710b0_2
libgcc-ng 11.2.0 h1234567_1
libgfortran-ng 13.2.0 h69a702a_0 conda-forge
libgfortran5 13.2.0 ha4646dd_0 conda-forge
libglib 2.68.4 h3e27bee_0 conda-forge
libgomp 11.2.0 h1234567_1
libiconv 1.17 h166bdaf_0 conda-forge
liblapack 3.9.0 16_linux64_openblas conda-forge
liblapacke 3.9.0 16_linux64_openblas conda-forge
libllvm11 11.1.0 hf817b99_2 conda-forge
libnghttp2 1.43.0 h812cca2_1 conda-forge
libogg 1.3.4 h7f98852_1 conda-forge
libopenblas 0.3.21 h043d6bf_0 http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libopencv 4.5.3 py36h9976982_2 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libpq 13.3 hd57d9b9_0 conda-forge
libprotobuf 3.16.0 h780b84a_0 conda-forge
libssh2 1.10.0 ha56f1ee_2 conda-forge
libstdcxx-ng 11.2.0 h1234567_1
libtiff 4.3.0 hf544144_1 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp-base 1.2.2 h7f98852_1 conda-forge
libxcb 1.13 h7f98852_1004 conda-forge
libxkbcommon 1.0.3 he3ba5ed_0 conda-forge
libxml2 2.9.12 h72842e0_0 conda-forge
lz4-c 1.9.3 h9c3ff4c_1 conda-forge
markdown 3.3.7 pypi_0 pypi
mysql-common 8.0.25 ha770c72_2 conda-forge
mysql-libs 8.0.25 hfa10184_2 conda-forge
ncurses 6.4 h6a678d5_0
nettle 3.6 he412f7d_0 conda-forge
nspr 4.32 h9c3ff4c_1 conda-forge
nss 3.69 hb5efdd6_1 conda-forge
numpy 1.19.5 pypi_0 pypi
opencv 4.5.3 py36h5fab9bb_2 conda-forge
openh264 2.1.1 h780b84a_0 conda-forge
openssl 1.1.1w h7f8727e_0
pcre 8.45 h9c3ff4c_0 conda-forge
pip 21.2.2 py36h06a4308_0
pixman 0.40.0 h36c2ea0_0 conda-forge
protobuf 3.19.6 pypi_0 pypi
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
py-opencv 4.5.3 py36he75451f_2 conda-forge
python 3.6.13 h12debd9_1
python_abi 3.6 2_cp36m conda-forge
pyyaml 6.0.1 pypi_0 pypi
qt 5.12.9 hda022c4_4 conda-forge
readline 8.2 h5eee18b_0
scipy 1.5.4 pypi_0 pypi
setuptools 58.0.4 py36h06a4308_0
six 1.17.0 pypi_0 pypi
sqlite 3.45.3 h5eee18b_0
tensorboard 1.8.0 pypi_0 pypi
tensorflow-gpu 1.8.0 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
tk 8.6.14 h39e8969_0
typing-extensions 4.1.1 pypi_0 pypi
werkzeug 2.0.3 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0
x264 1!161.3030 h7f98852_1 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.0.10 h7f98852_0 conda-forge
xorg-libsm 1.2.3 hd9c2040_1000 conda-forge
xorg-libx11 1.7.2 h7f98852_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h7f98852_1 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.6.4 h5eee18b_1
zipp 3.6.0 pypi_0 pypi
zlib 1.2.13 h5eee18b_1
zstd 1.5.0 ha95c52a_0 conda-forge

Could you please tell me how I can solve this problem? Thank you very much.