DevOps

Table of Contents

1. nodejs ubuntu
2. docker ubuntu
3. github ssh ubuntu
4. nginx ubuntu
5. haproxy ubuntu
6. let's encrypt ubuntu
7. amqp node sync
8. redis
9. postgres

nodejs-ubuntu

sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg

NODE_MAJOR=20
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list

sudo apt-get update
sudo apt-get install nodejs -y

node --version

improvement

sudo apt-get install build-essential

docker-ubuntu

Update the apt package index:

sudo apt-get update

Install packages to allow apt to use a repository over HTTPS:

sudo apt-get install \
 apt-transport-https \
 ca-certificates \
 curl \
 gnupg \
 lsb-release

Add Docker’s official GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Use the following command to set up the stable repository:

echo \
    "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Update the apt package index again, and install the latest version of Docker Engine and containerd:

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

Verify that Docker CE is installed correctly by running the hello-world image:

sudo docker run hello-world

if Dockerize Django Application

Dockerfile(django uwsgi):

# Use a Python base image
FROM python:3.8

# Set the working directory
WORKDIR /app

# Install uWSGI
RUN pip install uwsgi

# Copy the requirements file and install dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy the Django project code
COPY . .

# Expose the port on which uWSGI will listen
EXPOSE 8000

# Start uWSGI with the specified options
CMD ["uwsgi", "--http", "0.0.0.0:8000", "--module", "myapp.wsgi", "--processes", "4", "--threads", "2"]

Next, create a requirements.txt file in your project directory, listing your application's dependencies. For instance, if your Django app uses Django 3.2 and Gunicorn, your requirements.txt should contain:

Django==3.2
gunicorn

Build your Docker image:

sudo docker build -t my_django_app .

After the build completes, list your images with:

sudo docker images

run docker:

sudo docker run -d -p 8000:8000 testproject_web

or (following option)

sudo docker run -it -p 8000:8000 testproject_web

Stop a Container: To stop the running container, use the docker stop command followed by the container ID:

sudo docker stop 2ff3746e3305

Start a Container: If the container has been stopped, you can start it again using the docker start command:

sudo docker start 2ff3746e3305

View Container Logs: You can view the logs generated by the container using the docker logs command followed by the container ID:

sudo docker logs 2ff3746e3305

Inspect Container Details: To retrieve detailed information about the container, including its configuration and network details, use the docker inspect command:

sudo docker inspect 2ff3746e3305

github-ubuntu

ssh-keygen -t ed25519 -C "example@email.com"

eval "$(ssh-agent -s)"

ssh-add ~/.ssh/id_ed25519

cat ~/.ssh/id_ed25519.pub

nginx-ubuntu

If you haven't installed Nginx yet, you can do it with:

sudo apt update
sudo apt install nginx

You need to create a Nginx configuration file in /etc/nginx/sites-available/ directory. You can name it api.test.com. Use nano or vi or your favorite text editor to create and edit the file:

sudo nano /etc/nginx/sites-available/api.test.com

Paste the following into the file, replace api.test.com with your domain name:

server {
    listen 80;
    server_name api.test.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

To enable your site, you need to create a symbolic link from your site configuration in sites-available to the sites-enabled directory, which Nginx reads from during startup.

sudo ln -s /etc/nginx/sites-available/api.test.com /etc/nginx/sites-enabled/

(if exist)

sudo rm /etc/nginx/sites-enabled/api.test.com

Test and Restart Nginx

sudo nginx -t

sudo systemctl restart nginx

Enable Nginx auto-start when reboot

sudo systemctl enable nginx

to verify

sudo systemctl is-enabled nginx

haproxy-ubuntu

Install HAProxy: Update your package list and install HAProxy. Ubuntu's default repositories provide HAProxy, but if you need a specific version, you might have to add a dedicated PPA.

sudo apt update
sudo apt install haproxy

If you require a specific version, like 2.6, you can use a PPA:

sudo add-apt-repository ppa:vbernat/haproxy-2.6 -y
sudo apt update
sudo apt install -y haproxy=2.6.*

Configure HAProxy:

vim /etc/haproxy/haproxy.cfg

or

nano /etc/haproxy/haproxy.cfg

Example seeting:

# Frontend for RabbitMQ's management interface
frontend rabbitmq_mgmt_frontend
    bind *:15672
    default_backend rabbitmq_mgmt_backend

# Backend for RabbitMQ's management interface
backend rabbitmq_mgmt_backend
    mode http
    option forwardfor
    server rabbitmq1 <your-rabbitmq-server-ip>:15672 check

# Frontend for RabbitMQ's AMQP service
frontend rabbitmq_amqp_frontend
    bind *:5672
    default_backend rabbitmq_amqp_backend

# Backend for RabbitMQ's AMQP service
backend rabbitmq_amqp_backend
    mode tcp
    option tcplog
    server rabbitmq1 <your-rabbitmq-server-ip>:5672 check

restart and enable

sudo systemctl restart haproxy
sudo systemctl enable haproxy

lets-encrypt-ubuntu

First, make sure snapd is installed and up to date:

sudo apt update
sudo apt install snapd
sudo snap install core; sudo snap refresh core

Remove any old version of Certbot:

sudo apt-get remove certbot

Install Certbot:

sudo snap install --classic certbot

Link Certbot to your $PATH:

sudo ln -s /snap/bin/certbot /usr/bin/certbot

Use Certbot to get a certificate:

sudo certbot --nginx

If the verification is successful, Certbot will ask how you'd like to configure your HTTPS settings:

• Secure: This setting will redirect all HTTP traffic to HTTPS.
• Flexible: This setting will not redirect HTTP traffic. Certbot will automatically renew your certificates before they expire. You can test automatic renewal with this command:

Check if the domain needs to renew

sudo certbot renew --dry-run

Start the timer

sudo systemctl enable snap.certbot.renew.timer
sudo systemctl start snap.certbot.renew.timer

Check renew timer works or not

systemctl list-timers | grep certbot

Check again

systemctl list-timers | grep certbot
# Wed 2020-01-01 11:35:00 UTC 16h left      n/a                         n/a                snap.certbot.renew.timer       snap.certbot.renew.service

Check all certificates

sudo certbot certificates

Auto renew will only renew the verification within one month, if you want to force it to renew, and please restart nginx after doing so

sudo certbot renew --force-renewal --cert-name xxx.xxx.xxx
sudo systemctl restart nginx

or if you want to re-applying a new certificate

sudo certbot certonly --force-renewal -d example.domian

DNS Verification way for let's encrypt

# Install Required Packages
sudo apt update
sudo apt install python3-certbot-dns-cloudflare jq

Cloudflare API Token Setup

Get Cloudflare token:
Login: https://dash.cloudflare.com/
Navigate: Profile → API Tokens → Create Token

Template: Custom token
Permissions:
- Zone:Zone:Read
- Zone:DNS:Edit
Zone Resources:
- Include: Specific zone → your.domain.com

Get Zone ID from domain dashboard (right sidebar)

Create Cloudflare Credentials File

sudo nano /etc/letsencrypt/cloudflare.ini

Content for cloudflare.ini

dns_cloudflare_api_token = YOUR_CLOUDFLARE_API_TOKEN

Set permissions

sudo chmod 600 /etc/letsencrypt/cloudflare.ini

Request Wildcard Certificate

sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
  --cert-name example-wildcard \
  -d example.com \
  -d "*.example.com"

Verify New Certificate

sudo certbot certificates

Backup configurations

sudo cp -r /etc/nginx/sites-available /etc/nginx/sites-available.backup

Update certificate paths in all files

sudo sed -i 's|/etc/letsencrypt/live/example.com/|/etc/letsencrypt/live/example-wildcard/|g' /etc/nginx/sites-available/example.com

sudo nginx -t
sudo systemctl reload nginx

curl -I https://example.com

sudo nano /etc/letsencrypt/renewal-hooks/pre/cleanup-dns.sh

#!/bin/bash
# Pre-renewal DNS cleanup script

CF_API_TOKEN="YOUR_CLOUDFLARE_API_TOKEN"
ZONE_ID="YOUR_ZONE_ID"

echo "$(date): Starting DNS cleanup" >> /var/log/cert-cleanup.log

# Clean up _acme-challenge TXT records
curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?type=TXT&name=_acme-challenge.example.com" \
  -H "Authorization: Bearer $CF_API_TOKEN" \
  -H "Content-Type: application/json" | \
  jq -r '.result[].id' | \
  while read record_id; do
    if [ ! -z "$record_id" ]; then
      curl -s -X DELETE "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$record_id" \
        -H "Authorization: Bearer $CF_API_TOKEN" >> /var/log/cert-cleanup.log 2>&1
      echo "$(date): Deleted record $record_id" >> /var/log/cert-cleanup.log
    fi
  done

sleep 10
echo "$(date): DNS cleanup completed" >> /var/log/cert-cleanup.log

If you haven't installed jq

sudo apt install jq

Set executable permissions

sudo chmod +x /etc/letsencrypt/renewal-hooks/pre/cleanup-dns.sh

Multi-Server Auto-Renewal Configuration - Check current timer

sudo systemctl list-timers certbot.timer
sudo systemctl cat certbot.timer

sudo systemctl edit certbot.timer

input the following for time, randomizedDelay for multiple server

[Timer]
OnCalendar=
OnCalendar=*-*-* 01:00:00
RandomizedDelaySec=1800

Apply timer changes

sudo systemctl daemon-reload
sudo systemctl restart certbot.timer
sudo systemctl list-timers certbot.timer

Testing and Verification - Test automatic renewal

sudo certbot renew --dry-run

Check cleanup logs

tail -f /var/log/cert-cleanup.log

Verify certificate details

sudo certbot certificates
openssl x509 -in /etc/letsencrypt/live/example-wildcard/fullchain.pem -noout -dates

Cleanup Old Certificate - After confirming everything works

sudo certbot delete --cert-name example.com

Django change password:

from django.contrib.auth import get_user_model

User = get_user_model()
user = User.objects.get(email='')

user.set_password('')
user.save()

find . -path "*/migrations/*.py" -not -name "__init__.py" -delete

find . -path "*/__pycache__/*.pyc"  -delete

amqp

Install from docker:

docker pull rabbitmq:management

docker run -d --hostname rabbit_host1 --name rabbitmq1 -p 15672:15672 -p 5672:5672 -e RABBITMQ_ERLANG_COOKIE='rabbitmq_cookie' --restart always rabbitmq:management
#docker run -d --hostname rabbit_host2 --name rabbitmq2 -p 5673:5672 --link rabbitmq1:rabbit_host1 -e RABBITMQ_ERLANG_COOKIE='rabbitmq_cookie' --restart always rabbitmq:management
#docker run -d --hostname rabbit_host3 --name rabbitmq3 -p 5674:5672 --link rabbitmq1:rabbit_host1 --link rabbitmq2:rabbit_host2 -e RABBITMQ_ERLANG_COOKIE='rabbitmq_cookie' --restart always rabbitmq:management
docker run -d --hostname rabbit_host4 --name rabbitmq4 -p 5675:5672 --link rabbitmq1:rabbit_host1 --link rabbitmq2:rabbit_host2 --link rabbitmq3:rabbit_host3 -e RABBITMQ_ERLANG_COOKIE='rabbitmq_cookie' --restart always rabbitmq:management

redis

Start Redis:

docker run --name redis-7.2.5 -d \
  -e REDIS_PASSWORD=yourpassword \
  --restart always \
  -p 6379:6379 \
  redis:7.2.5 \
  redis-server --requirepass yourpassword

Create cluster:

redis-cli --cluster create <node1-ip>:6379 <node2-ip>:6379 <node3-ip>:6379 --cluster-replicas 0 -a your_redis_password

Add new server to cluster:

redis-cli --cluster add-node NEW_NODE_IP:6379 EXISTING_NODE_IP:6379

Move hash shard

redis-cli --cluster reshard

check cluster node

redis-cli -h ip -p 6379 -a password CLUSTER NODES

check health

redis-cli -a pass cluster info

postgres

Start PostgresSQL Docker Server

docker run --name my_postgres \
    -e POSTGRES_USER=myuser \
    -e POSTGRES_PASSWORD=mysecretpassword \
    -p 1555:5432 \
    -v my_postgres_data:/var/lib/postgresql/data \
    --restart always \
    -d postgres:16.2

Check PostgreSQL Docker Container ID or Name

Run the following command to list all running containers:

docker ps

Find the CONTAINER ID or NAME of the PostgreSQL container, such as postgres-container.

---

Access the Container's Interactive Terminal

Run the following command to access the container:

docker exec -it my_postgres bash

Here, my_postgres is the container name or ID.

---

Log in to the PostgreSQL Database

Inside the container, use the psql tool to log in to PostgreSQL:

psql -U myuser

-U myuser specifies the default PostgreSQL superuser myuser. If you have another administrative user, replace it with the corresponding username.

---

Execute SQL Commands to Create User and Set Permissions

Follow the steps below to create a new user and set permissions:

Create a User

CREATE USER new_user WITH PASSWORD 'your_password';

Set Permissions

GRANT CONNECT ON DATABASE your_database TO new_user;
GRANT USAGE ON SCHEMA public TO new_user;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO new_user;

(Optional) Grant Superuser Privileges

If the new user requires superuser privileges:

ALTER USER new_user WITH SUPERUSER;

---

Exit PostgreSQL and the Container

Exit PostgreSQL

\q

Exit the Container

exit