Skip to content

Add x86_64 CUDA support with separate profile #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bhavishadawada wants to merge 1 commit into
base: notebook-fixes
Choose a base branch
from
Open

Add x86_64 CUDA support with separate profile #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions src/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
version: "3.0"
services:
# NVIDIA Jetson devices (Xavier, Orin) - ARM64 with Tegra GPU
backend_tegra_gpu_enabled:
network_mode: "host"
build: ./backend
Expand Down Expand Up @@ -45,6 +46,7 @@ services:
count: all
capabilities: [gpu]
profiles: [tegra]
# CPU-only systems (x86_64, ARM64) - no GPU acceleration
backend_generic:
network_mode: "host"
build: ./backend
Expand Down Expand Up @@ -82,6 +84,55 @@ services:
- GRPC_ENABLE_FORK_SUPPORT=1
- PYTHONPATH=/opt/tritonserver/backends/python:/
profiles: [generic]
# x86_64 systems with NVIDIA GPU (Tesla T4, RTX, etc.)
backend_x86_cuda:
network_mode: "host"
build: ./backend
image: flask-app
privileged: true
expose:
- "5000"
- "5443"
- "8000"
- "8001"
- "80002"
volumes:
- /aws_dda:/aws_dda
- /tmp:/tmp
- /dev/shm:/dev/shm
- /usr/local/cuda:/usr/local/cuda:ro
- /usr/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:ro
environment:
- DDA_SYSTEM_USER_ID=$DDA_SYSTEM_USER_ID
- DDA_SYSTEM_GROUP_ID=$DDA_SYSTEM_GROUP_ID
- DDA_ADMIN_USER_ID=$DDA_ADMIN_USER_ID
- DDA_ADMIN_GROUP_ID=$DDA_ADMIN_GROUP_ID
- AWS_REGION
- SVCUID
- AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT
- AWS_CONTAINER_AUTHORIZATION_TOKEN
- AWS_CONTAINER_CREDENTIALS_FULL_URI
- AWS_IOT_THING_NAME
- COMPONENT_WORK_PATH
- KERNEL_ROOT_PATH
- INFERENCE_COMPONENT_DECOMPRESED_PATH
- LOCAL_SERVER_COMPONENT_DECOMPRESSED_PATH
- JETSON_CUDA=$JETSON_CUDA
- JETSON_TENSORRT=$JETSON_TENSORRT
- CUDA_HOME=/usr/local/cuda
- LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
- GRPC_POLL_STRATEGY=poll
- GRPC_ENABLE_FORK_SUPPORT=1
- PYTHONPATH=/opt/tritonserver/backends/python:/
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
profiles: [x86_cuda]
# React web interface - runs on all configurations
frontend:
build: ./frontend
image: react-webapp
Expand Down
4 changes: 3 additions & 1 deletion src/host_scripts/get_nvidia_libs_versions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,9 +143,11 @@ echo "JETSON_TENSORRT=${JETSON_TENSORRT}" >> /tmp/.dda.env
if [ -f /sys/devices/soc0/soc_id ]; then
is_gpu=0
fi
#Use gpu if aarch64 and CUDA is present
#Use gpu profile based on architecture and CUDA presence
if [ $is_gpu -eq 1 ] && [ $arch = "aarch64" ]; then
echo DOCKER_PROFILE='tegra' >> /tmp/.dda.env
elif [ $is_gpu -eq 1 ] && [ $arch = "x86_64" ]; then
echo DOCKER_PROFILE='x86_cuda' >> /tmp/.dda.env
else
echo DOCKER_PROFILE='generic' >> /tmp/.dda.env
fi