diff --git a/docs/running_docker.md b/docs/running_docker.md index e5d43698..11fbb4fb 100644 --- a/docs/running_docker.md +++ b/docs/running_docker.md @@ -4,7 +4,7 @@ > This method is **not intended for production use**. > It serves only as a **proof of concept** for testing Docker images as part of an existing pipeline. > -> Currently, it has been tested with the `alma9-flp-node` image running the *readout* component. +> Currently, it was tested with the `alma9-flp-node` image running the *readout* component with CRU. --- @@ -45,55 +45,69 @@ For example, to run readout, modify the `_plain_command` section of [`readout.ya When running readout, we successfully used the following command inside the `alma9-flp-node` image: ```bash -sudo /usr/bin/docker run --name readout --replace \ - --user "$(id -u flp):$(id -u flp)" \ +sudo -E docker run --name readout --replace \ + --user flp -v /etc/group:/etc/group:ro -v /etc/passwd:/etc/passwd:ro \ + --privileged \ --network=host --ipc=host \ - -e O2_DETECTOR -e O2_PARTITION -e OCC_CONTROL_PORT \ - -e O2_SYSTEM -e O2_ROLE \ + -v /tmp:/tmp \ + -v /lib/modules/$(uname -r):/lib/modules/$(uname -r) \ + -e O2_DETECTOR -e O2_PARTITION -e OCC_CONTROL_PORT -e O2_SYSTEM -e O2_ROLE \ gitlab-registry.cern.ch/aliceo2group/dockerfiles/alma9-flp-node:2 \ /opt/o2/bin/o2-readout-exe ``` > 🧩 **Note** > We are not claiming that this is the most efficient way how to run this image, just that it works. +> +#### Explanation of command + +Let's explain all parts of the `docker run` command and what is their purpose. As RHEL uses Podman by default +instead of docker we are going to comment on Podman parameters, but docker should be equivalent or pretty close. + +- `sudo -E` switch to the rootful mode of Podman and pass all of the environment variables +- `docker run --name readout --replace` start container with the name readout and replace already existing container +- `--privileged` runs container with extended privileges and allows access to the devices and other resources +on host system with the same privileges as user running the container (`flp` in our case). Readout communicates +directly with CRUs and others connected via PCIe so we need to make these available inside the container. +- `--user flp -v /etc/group:/etc/group:ro -v /etc/passwd:/etc/passwd:ro` run container as flp user provide same user +and group settings as on a host machine. This is necessary as readout is using shared memory which is mapped under +flp user and needs access to [BAR](https://github.com/AliceO2Group/ReadoutCard?tab=readme-ov-file#bar-interface) +interfaces of CRU which belong to the `pda` group. We used this way of setting up user privileges and ids as there is +no guarantee that ids would match if we would hardwire user into docker image +- `--network=host` bind container's network to the host's. This allows services running outside of docker to communicate +with internals (eg. gRPC, IL, ... ). It should be changed to open just required ports. +- `--ipc=host` readout uses FairMQ which communicates through shared memory Inter-Process Communication. +As we are now running readout inside the docker with the rest of data distribution running bare metal we +need to have access to the hosting OS shared memory. This is the main requirement for running as rootful. +If we run container in rootless mode with this flag set, Podman switches into elevated privileges mode that blocks +other cli commands from the same user command until this command is finished unless running under root. +We tried to just bind `/dev/shm/` but readout failed on permission errors. +- `-v /tmp:/tmp` binds `/tmp` of host to the container for monitoring purposes. +- `-v /lib/modules/$(uname -r):/lib/modules/$(uname -r)` binds folder with host's kernel modules for the usage of PDA +used by readout itself. +- `e ...` pass environment variables required by readout to the container #### Environment Variables -To identify all required environment variables: +--- + +## Tips and Tricks + +- To identify all required environment variables: 1. Open the **ECS GUI**. 2. Go to the **Environment Details** page for the relevant task. 3. Review the variables defined there — these match those used when running the binary outside Docker. -#### Shared Memory Communication - -To enable shared memory communication between processes, add the `--ipc=host` flag when running the container. -However, doing so requires **elevated privileges**. - -While **Podman** can run without root privileges, it pauses other Podman processes for the same user. -This means commands like `podman ps -a` or starting multiple containers in parallel will not work. - -Therefore, you should run containers using the same user as the rest of the pipeline: - -```bash ---user "$(id -u flp):$(id -u flp)" -``` - -This ensures shared memory segments are created under the same user context. - ---- - -## Tips and Tricks - -* Production systems running RHEL do not install native Docker via: +- Production systems running RHEL do not install native Docker via: ```bash dnf install docker ``` - Instead, they use **Podman**, which emulates Docker’s behavior but may differ in certain aspects. + Instead, they use Podman, which emulates Docker’s behavior but may differ in certain aspects. -* To check whether ECS has started a container, run: +- To check whether ECS has started a container, run: ```bash docker ps -a @@ -107,7 +121,7 @@ This ensures shared memory segments are created under the same user context. > su - flp > ``` -* To view container logs directly from Docker: +- To view container logs directly from Docker: ```bash docker logs