diff --git a/content/en/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy.md b/content/en/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy.md new file mode 100644 index 00000000..d651b5e9 --- /dev/null +++ b/content/en/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy.md @@ -0,0 +1,29 @@ +--- +title: 6.6.4 Kubelet Proxy +date: 2025-08-22T13:00:03+08:00 +description: Koupleless Module Controller V2 Kubelet Proxy +weight: 930 +--- + +## Kubelet Proxy + +The Kubelet Proxy is an enhanced feature of Module Controller V2 on the K8s side. +It allows users to interact directly with Module Controller V2 using the ``kubectl`` tool, +providing an operational experience similar to the native K8s Kubelet. + +
+ +

Logs command schematic

+
+ +## Iteration Plan + +The adaptation will be carried out in two phases: + +- [x] Use the proxy solution to provide logs capability for modules deployed in the Pod base -> **Completed** +- [ ] Ensure semantic consistency and implement logs capability through tunnel or arklet for smooth transition -> * + *Planned** + +## Notes + +Currently, only the logs capability is implemented, and the base must be deployed in the K8s cluster. diff --git a/content/en/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy.md b/content/en/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy.md new file mode 100644 index 00000000..e07c3891 --- /dev/null +++ b/content/en/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy.md @@ -0,0 +1,193 @@ +--- +title: 5.7 Enable Kubelet Proxy +date: 2025-08-22T13:00:03+08:00 +description: How to enable Koupleless Module Controller V2 Kubelet Proxy +weight: 1100 +--- + +## Kubelet Proxy + +Kubelet Proxy is an enhanced feature of Module Controller V2 on the K8s side. +It allows users to interact directly with Module Controller V2 using the ``kubectl`` tool, +providing an operational experience similar to the native K8s Kubelet. + +For design details, please refer to +the [documentation](/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy). + +## Enable Kubelet Proxy + +0. Deploy cert-manager to manage certificate generation and rotation + cert-manager is a Kubernetes plugin for automating the management and rotation of TLS certificates. It helps generate + and manage TLS certificates used for the Kubelet Proxy. + Please refer to the [cert-manager documentation](https://cert-manager.io/docs/installation/) for installation + instructions. + Here is a simple installation example (v1.18.2): + +```bash +kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml +``` + +After successful deployment, deploy the corresponding Issuer and Certificate: + +- To create Issuer + +```yaml +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: virtual-kubelet-issuer +spec: + selfSigned: {} +``` + +- To create Cert + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: virtual-kubelet-cert +spec: + secretName: virtual-kubelet-tls # secretName: virtual-kubelet-tls # The name of the Secret where the certificate is stored, which will be used later in the ModuleController + duration: 2160h # 90 days + renewBefore: 360h # renew 15 days before expiration + issuerRef: + name: virtual-kubelet-issuer # Reference to the above Issuer + kind: ClusterIssuer + commonName: koupleless-virtual-kubelet # Common Name + usages: + - server auth + - digital signature + - key encipherment +``` + +After creation, you can use the following command to check whether the certificate secret was generated successfully: + +If the output is similar to the following, the certificate has been generated successfully: + +```bash +kubectl get secret virtual-kubelet-tls +``` + +If the output is similar to the following, the certificate has been generated successfully: + +``` +NAME TYPE DATA AGE +virtual-kubelet-tls kubernetes.io/tls 3 1m +``` + +1. Add `pods/log` permission to the Role + +```yaml +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: virtual-kubelet-role +rules: + - apiGroups: [""] # "" indicates the core API group + resources: ["pods" , "pods/status", "pods/spec","nodes", "nodes/status", "events", "pods/log"] + verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] + - apiGroups: [ "apps" ] + resources: [ "deployments", "deployments/status", "deployments/spec", "daemonSets", "daemonSets/status", "daemonSets/spec" ] + verbs: [ "get", "watch", "list" ] + - apiGroups: [""] # "" indicates the core API group + resources: ["configmaps", "secrets", "services"] + verbs: ["get", "watch", "list"] + - apiGroups: ["coordination.k8s.io"] # "" indicates the core API group + resources: ["leases"] + verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] +``` + +2. Create a Service for the ModuleController deployment + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: module-controller + namespace: default + labels: + app: module-controller + virtual-kubelet.koupleless.io/kubelet-proxy-service: "true" # Necessary, indicates that this Service is used for Kubelet Proxy +spec: + selector: + app: module-controller + ports: + - name: httptunnel # If HTTP tunneling is not enabled, please remove this port + port: 7777 + targetPort: 7777 + - name: kubelet-proxy # Kubelet Proxy port + port: 10250 + type: ClusterIP +``` + +3. Modify the ENV configuration of ModuleController + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: module-controller +spec: + replicas: 1 + selector: + matchLabels: + app: module-controller + template: + metadata: + labels: + app: module-controller + spec: + serviceAccountName: virtual-kubelet + volumes: + - name: tls-certs + secret: + secretName: virtual-kubelet-tls # Necessary, mount the TLS certificate generated by cert-manager + containers: + - name: module-controller + image: serverless-registry.cn-shanghai.cr.aliyuncs.com/opensource/release/module-controller-v2: # Please replace with the actual version number, e.g., v2.1.4 + imagePullPolicy: IfNotPresent + resources: + limits: + cpu: "1000m" + memory: "400Mi" + ports: + - name: httptunnel # If HTTP tunneling is not enabled, please remove this port + containerPort: 7777 + - name: kubelet-proxy # Kubelet Proxy port + containerPort: 10250 + env: + - name: ENABLE_HTTP_TUNNEL + value: "true" + - name: NAMESPACE # Necessary, the namespace where ModuleController is deployed + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: KUBELET_PROXY_ENABLED # Necessary, enable Kubelet Proxy + value: "true" + volumeMounts: # Necessary, mount the TLS certificate generated by cert-manager + - name: tls-certs + mountPath: /etc/virtual-kubelet/tls + readOnly: true +``` + +## Verify Kubelet Proxy + +Assume that a module named `biz1-web-single-host` has been deployed and the Module Controller has enabled the Kubelet +Proxy. + +``` +NAME READY STATUS RESTARTS AGE +base-76d79d8599-f64jt 1/1 Running 0 13d +biz1-web-single-host-786dfc476f-qsp7q 1/1 Running 0 7m40s +module-controller-59f7bb765-8w84l 1/1 Running 0 13d +``` + +At this point, you can directly access the module's logs using the kubectl command: + +```bash +kubectl logs --tail=50 biz1-web-single-host-786dfc476f-qsp7q +``` + +It is expected to see normal log output. If an error occurs, it may indicate that the Kubelet Proxy is not properly +configured or not enabled. diff --git a/content/en/docs/tutorials/module-operation-v2/module-controller-deployment.md b/content/en/docs/tutorials/module-operation-v2/module-controller-deployment.md index 887ceab0..18a5e8d0 100644 --- a/content/en/docs/tutorials/module-operation-v2/module-controller-deployment.md +++ b/content/en/docs/tutorials/module-operation-v2/module-controller-deployment.md @@ -73,6 +73,13 @@ Below are some configurable environment variables and their explanations: - **CLIENT_ID** - Meaning: Optional, Module Controller instance ID. need to be unique in one env, will generate a random UUID in default. +- **KUBELET_PROXY_ENABLED** + - Meaning: Flag to enable Kubelet proxy. If `true`, the Kubelet proxy will be enabled. For prerequisites to enable, + please refer to documentation [here](/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy/). + +- **KUBELET_PROXY_PORT** + - Meaning: Port for Kubelet proxy. Default is 10250. + ### Documentation Reference -For detailed structure and implementation, refer to the [documentation](/docs/contribution-guidelines/module-controller-v2/architecture/). \ No newline at end of file +For detailed structure and implementation, refer to the [documentation](/docs/contribution-guidelines/module-controller-v2/architecture/). diff --git a/content/zh-cn/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy.md b/content/zh-cn/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy.md new file mode 100644 index 00000000..0ad84b33 --- /dev/null +++ b/content/zh-cn/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy.md @@ -0,0 +1,27 @@ +--- +title: 6.6.4 Kubelet 代理 +date: 2025-08-22T13:00:03+08:00 +description: Koupleless Module Controller V2 Kubelet 代理 +weight: 930 +--- + +## Kubelet 代理 + +Kubelet 代理是 Module Controller V2 在 K8s 侧的增强功能,它允许用户通过 ``kubectl`` 工具直接与 Module Controller V2 +交互,提供类似于 K8s 原生 Kubelet 的操作体验。 + +
+ +

logs 命令示意图

+
+ +## 迭代计划 + +适配分两阶段进行: + +- [x] 使用 proxy 代理方案,为部署在 Pod 基座中的模块提供 logs 能力 -> **已完成** +- [ ] 在保证语义的前提下,通过 tunnel 或 arklet 实现 logs 能力,完成平滑切换 -> **规划中** + +## 注意事项 + +当前仅实现了 logs 能力,且基座必须部署在 K8s 集群中。 diff --git a/content/zh-cn/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy.md b/content/zh-cn/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy.md new file mode 100644 index 00000000..fa2786e0 --- /dev/null +++ b/content/zh-cn/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy.md @@ -0,0 +1,185 @@ +--- +title: 5.7 启用 Kubelet 代理 +date: 2025-08-22T13:00:03+08:00 +description: Koupleless Module Controller V2 Kubelet 代理的启用方式 +weight: 1100 +--- + +## Kubelet 代理 + +Kubelet 代理是 Module Controller V2 在 K8s 侧的增强功能,它允许用户通过 ``kubectl`` 工具直接与 Module Controller V2 交互, +提供类似于 K8s 原生 Kubelet 的操作体验。 + +设计请参考[文档](/docs/contribution-guidelines/module-controller-v2/virtual-kubelet-proxy) + +## 启用 Kubelet 代理 + +0. 部署 cert-manager 管理证书的生成与轮换 + cert-manager 是一个 Kubernetes 插件,用于自动化管理和轮换 TLS 证书。它可以帮助我们生成和管理用于 Kubelet 代理的 TLS 证书。 + 请参考 [cert-manager 文档](https://cert-manager.io/docs/installation/) 进行安装。 + 这里给出一个简单的安装示例(v1.18.2): + +```bash +kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.2/cert-manager.yaml +``` + +部署成功后,我们部署相应的 Issuer 以及 Certificate: + +- 创建 Issuer + +```yaml +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: virtual-kubelet-issuer +spec: + selfSigned: {} +``` + +- 创建 Cert + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: virtual-kubelet-cert +spec: + secretName: virtual-kubelet-tls # 证书存储的 Secret 名称,后续我们将会在 ModuleController 中使用 + duration: 2160h # 90天 + renewBefore: 360h # 15天前续期 + issuerRef: + name: virtual-kubelet-issuer # 上一步创建的 Issuer + kind: ClusterIssuer + commonName: koupleless-virtual-kubelet # 公共名称 + usages: + - server auth + - digital signature + - key encipherment +``` + +创建完毕后,可以通过以下命令查看证书密钥是否生成成功: + +```bash +kubectl get secret virtual-kubelet-tls +``` + +如果输出类似以下内容,说明证书生成成功: + +``` +NAME TYPE DATA AGE +virtual-kubelet-tls kubernetes.io/tls 3 1m +``` + +1. 为 Role 增加 `pods/log` 权限 + +```yaml +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: virtual-kubelet-role +rules: + - apiGroups: [""] # "" indicates the core API group + resources: ["pods" , "pods/status", "pods/spec","nodes", "nodes/status", "events", "pods/log"] + verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] + - apiGroups: [ "apps" ] + resources: [ "deployments", "deployments/status", "deployments/spec", "daemonSets", "daemonSets/status", "daemonSets/spec" ] + verbs: [ "get", "watch", "list" ] + - apiGroups: [""] # "" indicates the core API group + resources: ["configmaps", "secrets", "services"] + verbs: ["get", "watch", "list"] + - apiGroups: ["coordination.k8s.io"] # "" indicates the core API group + resources: ["leases"] + verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] +``` + +2. 为 ModuleController 部署创建 Service + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: module-controller + namespace: default + labels: + app: module-controller + virtual-kubelet.koupleless.io/kubelet-proxy-service: "true" # 必须,用于标识这是 Kubelet 的代理 Service +spec: + selector: + app: module-controller + ports: + - name: httptunnel # 运维 HTTP 管道端口,如果启用了 MQTT 管道,请将此端口删除 + port: 7777 + targetPort: 7777 + - name: kubelet-proxy # Kubelet 代理端口,与 ModuleController ENV 中的 KUBELET_PROXY_PORT 保持一致 + port: 10250 + type: ClusterIP +``` + +3. 修改 ModuleController 的 ENV 配置 + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: module-controller +spec: + replicas: 1 + selector: + matchLabels: + app: module-controller + template: + metadata: + labels: + app: module-controller + spec: + serviceAccountName: virtual-kubelet + volumes: + - name: tls-certs + secret: + secretName: virtual-kubelet-tls # 必须,挂载前面创建的 TLS 证书 + containers: + - name: module-controller + image: serverless-registry.cn-shanghai.cr.aliyuncs.com/opensource/release/module-controller-v2:<版本号> + imagePullPolicy: IfNotPresent + resources: + limits: + cpu: "1000m" + memory: "400Mi" + ports: + - name: httptunnel # 如果未启用 HTTP 管道,请将此端口删除 + containerPort: 7777 + - name: kubelet-proxy # Kubelet 代理端口 + containerPort: 10250 + env: + - name: ENABLE_HTTP_TUNNEL + value: "true" + - name: NAMESPACE # 必须,用于标识 ModuleController Pod 所在的命名空间 + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: KUBELET_PROXY_ENABLED # 必须,启用 Kubelet 代理 + value: "true" + volumeMounts: # 必须,挂载 TLS 证书 + - name: tls-certs + mountPath: /etc/virtual-kubelet/tls + readOnly: true +``` + +## 验证 Kubelet 代理 + +假设目前已部署了名为 `biz1-web-single-host` 的模块,并且 Module Controller 启用了 Kubelet 代理。 + +``` +NAME READY STATUS RESTARTS AGE +base-76d79d8599-f64jt 1/1 Running 0 13d +biz1-web-single-host-786dfc476f-qsp7q 1/1 Running 0 7m40s +module-controller-59f7bb765-8w84l 1/1 Running 0 13d +``` + +此时,通过 kubectl 命令可以直接访问模块的日志: + +```bash +kubectl logs --tail=50 biz1-web-single-host-786dfc476f-qsp7q +``` + +期望会有正常的日志输出,如果报错,则可能是 Kubelet 代理未正确配置或未启用。 diff --git a/content/zh-cn/docs/tutorials/module-operation-v2/module-controller-deployment.md b/content/zh-cn/docs/tutorials/module-operation-v2/module-controller-deployment.md index 1a030e3f..9cb3d875 100644 --- a/content/zh-cn/docs/tutorials/module-operation-v2/module-controller-deployment.md +++ b/content/zh-cn/docs/tutorials/module-operation-v2/module-controller-deployment.md @@ -73,6 +73,13 @@ weight: 800 - **CLIENT_ID** - 含义: 可选配置,Module Controller 实例ID,需保证同环境中全局唯一,默认情况下会生成随机UUID +- **KUBELET_PROXY_ENABLED** +- 含义: Kubelet 代理启用标志,若为 `true`,将启用 kubelet + 代理。启用方式,请参考[文档](/docs/tutorials/module-operation-v2/enable-virtual-kubelet-proxy/)。 + +- **KUBELET_PROXY_PORT** +- 含义: Kubelet 代理端口,当 KUBELET_PROXY_ENABLED 设置为 `true` 时生效,默认值为 10250。 + ### 文档参考 具体的结构和实现介绍请参考[文档](/docs/contribution-guidelines/module-controller-v2/architecture/) diff --git a/static/img/module-controller-v2/kubelet_proxy_sequence_diagram.png b/static/img/module-controller-v2/kubelet_proxy_sequence_diagram.png new file mode 100644 index 00000000..b995ce17 Binary files /dev/null and b/static/img/module-controller-v2/kubelet_proxy_sequence_diagram.png differ