Add x86_64 CUDA support with separate profile #13
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added x86_64 NVIDIA GPU support to resolve CUDA runtime library errors: New Docker Compose Profile: Added backend_x86_cuda service for x86_64 systems with NVIDIA GPUs. Mounts CUDA libraries from host system (/usr/local/cuda, /usr/lib/x86_64-linux-gnu). Sets CUDA environment variables (CUDA_HOME, LD_LIBRARY_PATH). Enables NVIDIA GPU device access for Docker containers
Enhanced Hardware Detection: Updated get_nvidia_libs_versions.sh to automatically detect x86_64 + CUDA systems
Added logic to set DOCKER_PROFILE='x86_cuda' for x86_64 systems with GPU libraries
Maintains existing detection for Jetson (tegra) and CPU-only (generic) systems
Documentation: Added clear comments to Docker Compose services explaining target hardware:
backend_tegra_gpu_enabled: NVIDIA Jetson devices (Xavier, Orin) - ARM64 with Tegra GPU
backend_generic: CPU-only systems - no GPU acceleration
backend_x86_cuda: x86_64 systems with NVIDIA GPU (Tesla T4, RTX, etc.)
Problem Solved: Compiled models expecting CUDA 10.2 runtime libraries now have access to host system CUDA installation, eliminating "libcudart.so.10.2: cannot open shared object file" errors.
Testing: Automatic profile selection ensures correct configuration based on detected hardware without manual intervention.