Skip to content

Migrate Backend to JetPack 6 with Python 3.10 #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rajjainl wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Migrate Backend to JetPack 6 with Python 3.10 #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 66 additions & 32 deletions src/backend/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,48 +1,83 @@
ARG OS
# JetPack 6 Compatible Dockerfile
# - Ubuntu 22.04 base
# - Python 3.10 (native to JetPack 6)
# - CUDA 12.6, TensorRT 10.3, cuDNN 9.3 compatibility

ARG OS=22.04
FROM public.ecr.aws/ubuntu/ubuntu:${OS}
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y software-properties-common wget build-essential libffi-dev zlib1g-dev libssl-dev libsqlite3-dev gdb
RUN apt-get update
RUN wget https://www.python.org/ftp/python/3.9.23/Python-3.9.23.tgz
RUN tar -xzf Python-3.9.23.tgz
RUN cd Python-3.9.23 && \
./configure --enable-optimizations --enable-shared && \
make -j$(nproc) && \
make install
RUN find / -name "libpython3.9.so.1.0" -print
RUN update-alternatives --install /usr/local/bin/python3 python3 /usr/local/bin/python3.9 1 \
&& update-alternatives --set python3 /usr/local/bin/python3.9

# Install system dependencies
RUN apt-get update && apt-get install -y \
software-properties-common \
wget \
build-essential \
libffi-dev \
zlib1g-dev \
libssl-dev \
libsqlite3-dev \
gdb \
&& rm -rf /var/lib/apt/lists/*

# Install Python 3.10 (JetPack 6 native version)
RUN apt-get update && apt-get install -y \
python3.10 \
python3.10-dev \
python3.10-venv \
python3-pip \
&& rm -rf /var/lib/apt/lists/*

# Set Python 3.10 as default
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1 \
&& update-alternatives --set python3 /usr/bin/python3.10 \
&& ln -sf /usr/bin/pip3 /usr/bin/pip

# Verify Python installation
RUN python3 --version && pip3 --version

# Install GCC 11 for compilation compatibility
RUN add-apt-repository ppa:ubuntu-toolchain-r/test -y && \
apt-get update && \
apt install gcc-11 g++-11 -y && \
rm /usr/bin/gcc && \
rm /usr/bin/g++ && \
rm -f /usr/bin/gcc /usr/bin/g++ && \
ln -s /usr/bin/gcc-11 /usr/bin/gcc && \
ln -s /usr/bin/g++-11 /usr/bin/g++
ln -s /usr/bin/g++-11 /usr/bin/g++ && \
rm -rf /var/lib/apt/lists/*

# Update ncurses packages
RUN apt-get update && \
apt-get install --only-upgrade -y ncurses-base ncurses-bin && \
rm -rf /var/lib/apt/lists/*

RUN apt-get install --only-upgrade ncurses-base
RUN apt-get install --only-upgrade ncurses-bin
# Install additional dependencies
RUN apt-get update && apt-get install -y libb64-0d && rm -rf /var/lib/apt/lists/*

# Copy and install Python requirements
COPY requirements.txt ./
RUN pip3 install --no-cache-dir --upgrade pip setuptools wheel && \
pip3 install --no-cache-dir -r requirements.txt

# Copy and setup edge management components
COPY edge_ml1_p_camera_management ./edge_ml1_p_camera_management
COPY edgeagent ./edgeagent
RUN ./edge_ml1_p_camera_management/prereqs_install.sh
RUN ./edge_ml1_p_camera_management/install_aravis.sh
RUN python3.9 -m grpc_tools.protoc --proto_path=./edgeagent --python_out=. --grpc_python_out=. edgeagent/edge-agent.proto
RUN apt update -y
RUN apt-get install libssl1.1 -y
RUN apt update -y
RUN apt install libexif12 libcurl4 libarchive13 gstreamer1.0-tools gstreamer1.0-libav ffmpeg -y
RUN if [ "$OS" = "18.04" ] ; then \
apt install libavcodec-extra57i -y; \
fi
RUN python3 -m grpc_tools.protoc --proto_path=./edgeagent --python_out=. --grpc_python_out=. edgeagent/edge-agent.proto

RUN find / -name "libpython3.9.so.1.0" -print
# Install multimedia and GStreamer dependencies (JetPack 6 compatible)
RUN apt-get update -y && \
apt-get install -y \
libexif12 \
libcurl4 \
libarchive13 \
gstreamer1.0-tools \
gstreamer1.0-libav \
ffmpeg \
&& rm -rf /var/lib/apt/lists/*

RUN apt clean -y

# Copy application code
COPY app.py ./
COPY logging.conf ./
COPY dao ./dao
Expand All @@ -64,15 +99,14 @@ COPY alembic.ini ./
COPY dda_triton ./dda_triton
COPY lyra_anomalies_mask_utils ./lyra_anomalies_mask_utils
COPY lyra_science_processing_utils ./lyra_science_processing_utils
CMD ["python3.9", "app.py"]

# Temporary to fix CVE-2024-24806 and CVE-2024-0553 until slim-buster base image is fixed
# Set default command
CMD ["python3", "app.py"]

# Security updates
RUN apt-get update \
# Install build tools (if needed for compiling)
&& apt-get install -y --no-install-recommends build-essential \
# Install tools for managing libraries
libgnutls28-dev libuv1 \
# Clean up to reduce image size
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

Expand All @@ -81,7 +115,7 @@ RUN apt-get update \
COPY edgemlsdk ./edgemlsdk
RUN ./edge_ml1_p_camera_management/install_edgemlsdk.sh
COPY dlr_disable_phone_home.py ./
RUN python3.9 dlr_disable_phone_home.py
RUN python3 dlr_disable_phone_home.py

#TODO:change to user to scope the permission. Currently needed to persist the TinyDB data
USER root
92 changes: 72 additions & 20 deletions src/backend/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,84 @@
# JetPack 6 Compatible Requirements
# Python 3.10 (native to JetPack 6 / Ubuntu 22.04)
# CUDA 12.6, TensorRT 10.3 compatible

# Build tools
setuptools
wheel
meson

# Computer Vision
opencv-python

# HTTP/Networking
urllib3==2.2.3
requests==2.32.3

# Date/Time utilities
python-dateutil==2.8.2
Jetson.GPIO==2.0.21
awsiotsdk==1.11.9
marshmallow==3.19.0
pyyaml==5.4.1
grpcio==1.56.2
grpcio-tools==1.51.1
jsonschema==4.17.3
numpy==1.24.3

# Jetson GPIO - verified for JetPack 6 and Python 3.10
# Note: On JetPack 6 Orin, may need to install via: pip install Jetson.GPIO
Jetson.GPIO==2.1.0

# AWS IoT SDK
awsiotsdk==1.21.0

# Data Serialization
marshmallow==3.21.0

# YAML parsing - upgraded for Python 3.10 compatibility
pyyaml==6.0.1

# gRPC - aligned versions for compatibility
grpcio==1.62.0
grpcio-tools==1.62.0

# Schema validation
jsonschema==4.21.0

# NumPy - upgraded for Python 3.10 optimization
numpy==1.26.4

# Image processing
Pillow==10.3.0
pycairo==1.23.0
PyGObject==3.42.2

# Cairo/GTK bindings
pycairo==1.26.0
PyGObject==3.48.0

# Hardware peripherals
python-periphery==2.4.1
psutil==5.9.5
fastapi==0.109.2
uvicorn==0.23.2
pydantic==2.1.1

# System utilities
psutil==5.9.8

# Web Framework
fastapi==0.110.0
uvicorn==0.28.0
pydantic==2.6.0

# Neo Deep Learning Runtime (DLR)
# Note: For JetPack 6, may need to build from source or use NVIDIA's pre-built wheel
# Check: https://github.com/neo-ai/neo-ai-dlr for JetPack 6 compatibility
dlr==1.10.0
asgi-correlation-id==4.2.0
structlog==22.1.0

# ASGI middleware
asgi-correlation-id==4.3.0

# Structured logging
structlog==24.1.0

# Deprecation warnings
# https://pypi.org/project/Deprecated/
Deprecated==1.2.14
sqlalchemy==2.0.21
alembic==1.12.1
httpx==0.24.1
scikit-learn==1.0.2

# Database ORM
sqlalchemy==2.0.28
alembic==1.13.1

# HTTP client
httpx==0.27.0

# Machine Learning - upgraded for Python 3.10 support
scikit-learn==1.4.0