Creating Docker Compose for Container

[Mcrich23]

Hello All,
I have been working to bring Docker Compose to Container, aka Container Compose. As I build this, I would like to share my thoughts on pre-requisites to having a fantastic compose experience, and get thoughts from the community on how this should be architected.

Currently, Compose supports most configuration options (and almost all common configuration options). But here is what I believe needs to happen to make compose strong and reliable:

• IP Consistency: I believe this was in the first version of Container, but it has since gone away. IPs not staying consistent between runs makes it hard to build a project dependent on a container, because it is not always the same IP. This means that it has to be updated each time it goes up.
• Further Networking Support: I'm guessing that this is already in the works, but it appears that the most common option my version of compose is currently lacking real support in is networking due to limitations in the CLI.
• Update Containers Without Removing Them: I think it might be handy (and more resource efficient) to allow users to update configurations without having to entirely remove the container and re-add it.

This is all that I can think of right now. I would love to understand everyone else's thoughts and ideas for compose. For more technical details, you can look at my pull request: #239.

Thanks!

[Mcrich23]

The initial version of Compose I made relied on calling into the shell environment. I just refactored most of it to actually call into the internal tools, but I cannot figure out the best way to handle multiple containers sending logs at once if someone streams it in. So that is currently using a shell command.

Curious if anyone can create a cool solution to this problem!

[jglogan]

Thanks for putting this up!

Regarding IP consistency, this was put done by design. Each new container gets a new MAC address (assigned by vmnet) and a new IP address (assigned by the network helper, using a LRU policy).

This was done because in practice, rapidly reusing IP addresses with different MAC addresses can cause temporary networking problems due to ARP tables being incorrect.

I'll add a note to the issue that requests manually assigned IP addresses suggesting that the scope include MAC address assignment as well.

I'm still not clear for the need for control over IP assignment in a Compose-like product. Hardcoding IP addresses in any service configuration feels like an anti-pattern. I've only ever written Compose configurations or Kubernetes manifests using service names, precisely because IP addresses are generally not stable over time.

[jglogan]

That's a good argument for using the discussion here to test out designs. What would your compose file look like for a reverse proxy service in front of a single API service, and what application configuration would you set up for each service?

[Mcrich23]

Well I might run nginx bare metal and have it hook into my containers.

[jglogan]

Would something like this work (courtesy ChatGPT)

version: '3.8'

services:
  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - api-a
      - api-b

  api-a:
    image: your-api-a-image
    environment:
      - DATABASE_URL=postgres://user:password@db:5432/appdb
    depends_on:
      - db

  api-b:
    image: your-api-b-image

  db:
    image: postgres:15
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

nginx.conf:

events {}

http {
    upstream api_a {
        server api-a:80;
    }

    upstream api_b {
        server api-b:80;
    }

    server {
        listen 80;

        location /api-a/ {
            proxy_pass http://api_a/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        location /api-b/ {
            proxy_pass http://api_b/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

[wagneramichael]

Normally with Nginx on Docker and Docker-Compose you would use the service name. Each service name gets a dns entry on the Docker network dns linked to that compose and you would config Nginx to use the Docker dns server.

Nginx config for the docker dns would be:

http {
resolver 127.0.0.11;
}

[Mcrich23]

Possibly with a custom dns. I would need to do a ton of testing. But for the sake of compose, I have already figured out a way to resolve it without having to touch dns if someone doesn't want to.

[jglogan]

@Mcrich23 Could you rework your PR so that compose gets built and installed as a plugin instead of hardwiring it into the CLI? We want users to be able to opt into features like this, and it's a good test case for the nascent extension facility that's in container.

[Mcrich23]

Just finished making it work. I have moved the PR from draft back to open and I believe that a first iteration is ready to be implemented.

[jglogan]

Thanks for the updates!

I haven't tried building and using this but I'll give it a shot later this week.

I've indicated in the PR the next steps for modularizing it into a plugin. Basically all that's needed is to take what you added to the Makefile and distribute it across three new targets:

• Copy the plugin pieces into a dedicated staging directory for packaging and sign the binary
• Create an installer package
• Install the installer package (using two strategies based on whether your running installer to put it in the true install root, or just running a build and installing it non-root into libexec in your project directory

The other thing to start looking at is the ComposeUp.swift file. That's a pretty big hunk of code - do you see any ways that its logic can be extracted into more easily maintainable and testable units?

One other question, how did you generate the codable types that map to the Compose format?

[Mcrich23]

We can map the functions into smaller chunks but they are mostly just single functions and may make sense to maintain as a single object. For the codable types, those were pulled from another early attempt at compose for container by @youroldmangaming.