40 add s3 instructions for juicefs

docs/crds/examples.md

@@ -69,10 +69,157 @@ For more information on using environment variables for Typesense consult the [o
 
 ## Use S3-Compliant Storage as Persistent Volumes
 
-As of **v0.3.6**, storage `accessMode` is introduced that supports `ReadWriteOnce` and `ReadWriteMany`. 
+As of **v0.3.6**, storage `accessMode` is introduced that supports `ReadWriteOnce` and `ReadWriteMany`, which means we can use local or remote S3-compliant object storage as the persistent store for the Typesense data volume.
 
-If you are running on [Open Telekom Cloud](https://www.open-telekom-cloud.com/en), you can take advantage of the additional annotations field `csi.storage.k8s.io/fstype` 
-that controls how an S3 bucket is mounted into a Kubernetes pod. 
+### Prerequisites/Dependencies
+
+* [JuiceFS CSI Driver](https://juicefs.com/docs/csi/introduction/)
+* JuiceFS Storage Class
+* Redis as JuiceFS metadata store
+
+:::warning
+
+1. When working with [Open Telekom Cloud Object Storage Service (OBS)](https://www.open-telekom-cloud.com/en/products-services/core-services/object-storage-service), [JuiceFS](https://juicefs.com/en/) and [Redis](https://redis.io/) **are not** required in order to mount S3 bucket in the Pods. The `everest-csi-provisioner` takes care of the whole life-cycle of provisioning and mount bucket as volumes
+via annotations. You can skip this section entirely.
+
+2. When working with [Amazon S3](https://aws.amazon.com/s3/), [Redis](https://redis.io/) **is not** required, as Amazon S3 comes with is own metadata endpoints.
+
+:::
+
+#### Install the JuiceFS CSI Storage Driver
+
+```bash
+kubectl apply -f https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s.yaml
+```
+
+#### Install a Redis Cluster
+
+```bash
+kubectl create namespace redis
+kubectl apply -f redis.yaml -n redis
+```
+
+```yaml title="redis.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+  name: redis-service
+  namespace: redis
+  labels:
+    app: redis
+spec:
+  ports:
+    - port: 6379
+  clusterIP: None
+  selector:
+    app: redis
+---
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: redis-config
+  namespace: redis
+  labels:
+    app: redis
+data:
+  master.conf: |
+    maxmemory 1024mb
+    maxmemory-policy allkeys-lru
+    maxclients 20000
+    timeout 300
+    appendonly no
+    dbfilename dump.rdb
+    dir /data
+  secondary.conf: |
+    slaveof redis-0.redis.redis 6379
+    maxmemory 1024mb
+    maxmemory-policy allkeys-lru
+    maxclients 20000
+    timeout 300
+    dir /data
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: redis
+  namespace: redis
+spec:
+  serviceName: "redis-service"
+  replicas: 1
+  selector:
+    matchLabels:
+      app: redis
+  template:
+    metadata:
+      labels:
+        app: redis
+    spec:
+      initContainers:
+      - name: init-redis
+        image: redis:7.2.4
+        command:
+        - bash
+        - "-c"
+        - |
+          set -ex
+          # Generate redis server-id from pod ordinal index.
+          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1
+          ordinal=${BASH_REMATCH[1]}
+          # Copy appropriate redis config files from config-map to respective directories.
+          if [[ $ordinal -eq 0 ]]; then
+            cp /mnt/master.conf /etc/redis-config.conf
+          else
+            cp /mnt/slave.conf /etc/redis-config.conf
+          fi
+        volumeMounts:
+        - name: redis-claim
+          mountPath: /etc
+        - name: config-map
+          mountPath: /mnt/
+      containers:
+      - name: redis
+        image: redis:7.2.4
+        ports:
+        - containerPort: 6379
+          name: redis
+        command:
+          - redis-server
+          - "/etc/redis-config.conf"
+        volumeMounts:
+        - name: redis-data
+          mountPath: /data
+        - name: redis-claim
+          mountPath: /etc
+      volumes:
+      - name: config-map
+        configMap:
+          name: redis-config                  
+  volumeClaimTemplates:
+  - metadata:
+      name: redis-claim
+    spec:
+      accessModes: [ "ReadWriteOnce" ]
+      resources:
+        requests:
+          storage: 10Gi
+  - metadata:
+      name: redis-data
+    spec:
+      accessModes: [ "ReadWriteOnce" ]
+      resources:
+        requests:
+          storage: 10Gi
+```
+
+:::danger
+This is by no means a production-ready Redis cluster nor a best-practice on how to deploy Redis on Kubernetes. It's just a quick spin of cluster in order to be able to work with JuiceFS.
+:::
+
+### Open Telekom Cloud OBS
+
+If you are running on [Open Telekom Cloud](https://www.open-telekom-cloud.com/en), you can take advantage of the additional annotations field `csi.storage.k8s.io/fstype`
+that controls how an S3 bucket is mounted into a Kubernetes pod.
 
 - Using `csi.storage.k8s.io/fstype: s3fs` mounts an [SFS Turbo, Scalable File System](https://www.open-telekom-cloud.com/en/products-services/core-services/scalable-file-service) bucket using the S3-compatible API, 
 which is useful when you want simple “filesystem-like” access but are okay with object-storage semantics that may not fully match POSIX behavior.
@@ -109,6 +256,124 @@ For more information about configuring Open Telekom Cloud `StorageClass` consult
 the [official documentation](https://docs.otc.t-systems.com/cloud-container-engine/umn/storage/storageclass.html).
 :::
 
+### Amazon Web Services S3
+
+#### Create a JuiceFS Storage Class for AWS-S3
+
+```bash
+kubectl create namespace juicefs-system
+kubectl apply -f juicefs-aws-sc.yaml -n juicefs-system
+```
+
+```yaml title="juicefs-aws-sc.yaml"
+apiVersion: v1
+kind: Secret
+metadata:
+  name: juicefs-aws-sc-secret
+  namespace: juicefs-system
+type: Opaque
+stringData:
+  name: "aws-s3-test"
+  metaurl: "redis://<bucket>.0001.use1.cache.amazonaws.com/3"
+  storage: "s3"
+  bucket: "https://<bucket>.s3.us-east-1.amazonaws.com"
+  access-key: "<access-key>"
+  secret-key: "<secret-key>"
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: juicefs-aws-sc
+provisioner: csi.juicefs.com
+reclaimPolicy: Retain
+volumeBindingMode: Immediate
+parameters:
+  csi.storage.k8s.io/node-publish-secret-name: juicefs-aws-sc-secret
+  csi.storage.k8s.io/node-publish-secret-namespace: juicefs-system
+  csi.storage.k8s.io/provisioner-secret-name: juicefs-aws-sc-secret
+  csi.storage.k8s.io/provisioner-secret-namespace: juicefs-system
+```
+
+
+```bash
+kubectl apply -f tsc.yaml
+```
+
+```yaml title="tsc.yaml"
+apiVersion: ts.opentelekomcloud.com/v1alpha1
+kind: TypesenseCluster
+metadata:
+  labels:
+    app.kubernetes.io/name: typesense-operator
+    app.kubernetes.io/managed-by: kustomize
+  name: tsc
+spec:
+  image: typesense/typesense:29
+  replicas: 3
+  storage:
+    size: 150Mi
+    storageClassName: juicefs-aws-sc
+    accessMode: ReadWriteMany
+```
+
+### Other (RustFS, SeaweedFS etc.)
+
+#### Create a JuiceFS Storage Class for an S3-compliant service
+
+```bash
+kubectl create namespace juicefs-system
+kubectl apply -f juicefs-sc.yaml -n juicefs-system
+```
+
+```yaml title="juicefs-sc.yaml"
+apiVersion: v1
+kind: Secret
+metadata:
+  name: juicefs-sc-secret
+  namespace: juicefs-system
+type: Opaque
+stringData:
+  name: "s3-test"
+  metaurl: "redis://redis-service.redis:6379/1"
+  storage: "s3"
+  bucket: "https://<bucket-url>"
+  access-key: "<access-key>"
+  secret-key: "<secret-key>"
+---
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: juicefs-sc
+provisioner: csi.juicefs.com
+reclaimPolicy: Retain
+volumeBindingMode: Immediate
+parameters:
+  csi.storage.k8s.io/node-publish-secret-name: juicefs-sc-secret
+  csi.storage.k8s.io/node-publish-secret-namespace: juicefs-system
+  csi.storage.k8s.io/provisioner-secret-name: juicefs-sc-secret
+  csi.storage.k8s.io/provisioner-secret-namespace: juicefs-system
+```
+
+```bash
+kubectl apply -f tsc.yaml
+```
+
+```yaml title="tsc.yaml"
+apiVersion: ts.opentelekomcloud.com/v1alpha1
+kind: TypesenseCluster
+metadata:
+  labels:
+    app.kubernetes.io/name: typesense-operator
+    app.kubernetes.io/managed-by: kustomize
+  name: tsc
+spec:
+  image: typesense/typesense:29
+  replicas: 3
+  storage:
+    size: 150Mi
+    storageClassName: juicefs-sc
+    accessMode: ReadWriteMany
+```
 
 ## Scrapers
 

docs/crds/index.mdx

@@ -69,14 +69,23 @@ Typesense Kubernetes Operator is controlling the lifecycle of multiple Typesense
 :::important
 As of **v0.3.6**, storage `accessMode` is introduced that supports `ReadWriteOnce` and `ReadWriteMany`. 
 
-- If you are running on [Open Telekom Cloud](https://www.open-telekom-cloud.com/en), you can take advantage of the additional annotations field `csi.storage.k8s.io/fstype` 
+* If you are running on [Open Telekom Cloud](https://www.open-telekom-cloud.com/en), you can take advantage of the additional annotations field `csi.storage.k8s.io/fstype` 
 that controls how an S3 bucket is mounted into a Kubernetes pod. Using `csi.storage.k8s.io/fstype: s3fs` mounts an [SFS Turbo, Scalable File System](https://www.open-telekom-cloud.com/en/products-services/core-services/scalable-file-service) bucket using the S3-compatible API, 
 which is useful when you want simple “filesystem-like” access but are okay with object-storage semantics that may not fully match POSIX behavior. 
 Using the annotation `csi.storage.k8s.io/fstype: obsfs` mounts the bucket using the native OBS filesystem driver and is generally the more “OBS-native” approach; 
 it avoids some S3FS-specific limitations and is often preferred when you want better integration with OBS. 
 One big advantage of using these annotations is that the respective S3 buckets can be provisioned dynamically by the CSI.
 
-- If you are running on-premises clusters with [democratic-csi](https://github.com/democratic-csi/democratic-csi) installed as the CSI, you can use now use `accessMode: ReadWriteMany` to provision NFS-backed persistent volumes to support Typesense data volumes.
+* If you are running on-premises clusters with [democratic-csi](https://github.com/democratic-csi/democratic-csi) installed as the CSI, you can use now use `accessMode: ReadWriteMany` to provision NFS-backed persistent volumes to support Typesense data volumes.
+
+* If you run on-premises clusters with custom S3-compatible storage (e.g., RustFS, SeaweedFS, JuiceFS, Ceph, CubeFS), you can now use `accessMode: ReadWriteMany` together 
+with the `juicefs-csi-driver` to mount S3 buckets as persistent volumes in pods as Typesense data volumes.
+
+* If you run on-premises clusters or in any cloud provider, you can now attach any remote S3-compatible storage (e.g., AWS S3, Azure Blob Storage etc.), using `accessMode: ReadWriteMany` together 
+with the `juicefs-csi-driver` to mount S3 buckets as persistent volumes in pods as Typesense data volumes.
+
+For more information about installing and configuring **JuiceFS** and **JuiceFS CSI Driver** consult [the official documentation](https://juicefs.com/en/).
+
 :::
 
 ### IngressSpec (optional)