[GH-ISSUE #1463] [DOCS]: Basic k8s kubernetes manifest. #934

New Issue

Closed

opened 2026-02-22 18:22:16 -05:00 by yindo · 28 comments

yindo commented 2026-02-22 18:22:16 -05:00

Owner

Copy Link

Copy Source

Originally created by @atljoseph on GitHub (May 20, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1463

Description

Thanks for the docs about deploying to various hosting providers. Can you add a little pixie dust outlining some k8s deployment specs? Docker docs help for sure, but would be cool to have a template.

Originally created by @atljoseph on GitHub (May 20, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1463 ### Description Thanks for the docs about deploying to various hosting providers. Can you add a little pixie dust outlining some k8s deployment specs? Docker docs help for sure, but would be cool to have a template.

yindo added the documentation label 2026-02-22 18:22:16 -05:00

yindo closed this issue 2026-02-22 18:22:16 -05:00

yindo commented 2026-02-22 18:22:16 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (May 20, 2024):

I'd have to delegate this to someone in the community who has more experience with doing this. We can add it to the community templates

@timothycarambat commented on GitHub (May 20, 2024): I'd have to delegate this to someone in the community who has more experience with doing this. We can add it to the [community templates](https://github.com/Mintplex-Labs/anything-llm/tree/master/cloud-deployments)

yindo commented 2026-02-22 18:22:16 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 20, 2024):

I'll post something here later today... Figured it out

@atljoseph commented on GitHub (May 20, 2024): I'll post something here later today... Figured it out

yindo commented 2026-02-22 18:22:16 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

Expect pr today actually. Lost some time to aws ebs drivers.

@atljoseph commented on GitHub (May 21, 2024): Expect pr today actually. Lost some time to aws ebs drivers.

yindo commented 2026-02-22 18:22:16 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

We were able to make it work mostly, but something is driving me absolutely crazy.
If there really is no example in the repo already, then that likely means this hasn't been done yet.
In that case, I'll list the manifest that we have aas well as the symptoms.
Hoping for a quick resolution, as this is a value-add all-around.

---
apiVersion: v1                                                                                                                                           
kind: PersistentVolume                                                                                                                                   
metadata:                                                                                                                                                
  name: anything-llm-volume                                                                                                                              
  annotations:                                                                                                                                           
    pv.beta.kubernetes.io/uid: "1000"                                                                                                                    
    pv.beta.kubernetes.io/gid: "1000"                                                                                                                    
spec:                                                                                                                                                    
  storageClassName: gp2                                                                                                                                  
  capacity:                                                                                                                                              
    storage: 5Gi                                                                                                                                        
  accessModes:                                                                                                                                           
    - ReadWriteOnce                                                                                                                                      
  awsElasticBlockStore:                                                                                                                                  
    volumeID: ec2-ebs-volume-uuid-from-aws-console                                                                                                                           
    fsType: ext4
  nodeAffinity:                                                                                                                                          
    required:                                                                                                                                            
      nodeSelectorTerms:                                                                                                                                 
      - matchExpressions:                                                                                                                                
        - key: topology.kubernetes.io/zone                                                                                                               
          operator: In                                                                                                                                   
          values:                                                                                                                                        
          - us-direction-123  
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: anything-llm-volume-claim
  namespace: "{{ namespace }}"
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: anything-llm
  namespace: "{{ namespace }}"
  labels:
    anything-llm: "true"
spec:
  selector:
    matchLabels:
      k8s-app: anything-llm
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0%
      maxUnavailable: 100%
  template:
    metadata:
      labels:
        anything-llm: "true"
        k8s-app: anything-llm
        app.kubernetes.io/name: anything-llm
        app.kubernetes.io/part-of: anything-llm
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: /metrics
        prometheus.io/port: "9090"
    spec:
      serviceAccountName: "default"
      terminationGracePeriodSeconds: 10
      securityContext:                                                                                                                                                              
        fsGroup: 1000
        runAsNonRoot: true                                                                                                                                                          
        runAsGroup: 1000
        runAsUser: 1000
      affinity:                                                                                                                                                                                                                                                                          
        nodeAffinity:                                                                                                                                                                                                                                                                    
          requiredDuringSchedulingIgnoredDuringExecution:                                                                                                                                                                                                                                
            nodeSelectorTerms:                                                                                                                                                                                                                                                           
            - matchExpressions:                                                                                                                                                                                                                                                          
              - key: topology.kubernetes.io/zone                                                                                                                                                                                                                                         
                operator: In                                                                                                                                                                                                                                                             
                values:                                                                                                                                                                                                                                                                  
                - us-direction-123  
      containers:
      - name: anything-llm
        resources:
          limits:
            memory: "1Gi"
            cpu: "500m"
          requests:
            memory: "512Mi"
            cpu: "250m"
        imagePullPolicy: IfNotPresent
        image: "mintplexlabs/anythingllm:master"
        securityContext:                     
          allowPrivilegeEscalation: true                                                                                                                                                                                                                                                 
          capabilities:                                                                                                                                                                                                                                                                  
            add:                                                                                                                                                                                                                                                                         
              - SYS_ADMIN                                                                                                                                                                                                                                                                
          runAsNonRoot: true                                                                                                                                                                                                                                                             
          runAsGroup: 1000                                                                                                                                                                                                                                                               
          runAsUser: 1000                                                                                                                                       
        command: # Specify a command to override the Dockerfile's ENTRYPOINT.
          - /bin/bash
          - -c
          - |
            # Some debug stuff...
            set -x -e
            sleep 3
            echo "AWS_REGION: $AWS_REGION"
            echo "SERVER_PORT: $SERVER_PORT"
            echo "NODE_ENV: $NODE_ENV"
            echo "STORAGE_DIR: $STORAGE_DIR"
            # The following was taken from the Dockerfile's ENTRYPOINT, since `command` overrides that.
            {
              cd /app/server/ &&
                npx prisma generate --schema=./prisma/schema.prisma &&
                npx prisma migrate deploy --schema=./prisma/schema.prisma &&
                node /app/server/index.js
              echo "Server process exited with status $?"
            } &
            { 
              node /app/collector/index.js
              echo "Collector process exited with status $?"
            } &
            wait -n
            exit $?
        readinessProbe:
          httpGet:
            path: /v1/api/health
            port: 8888
          initialDelaySeconds: 15
          periodSeconds: 5
          successThreshold: 2
        livenessProbe:
          httpGet:
            path: /v1/api/health
            port: 8888
          initialDelaySeconds: 15
          periodSeconds: 5
          failureThreshold: 3
        env:
          - name: AWS_REGION
            value: "{{ aws_region }}"
          - name: AWS_ACCESS_KEY_ID
            value: "{{ aws_access_id }}"
          - name: AWS_SECRET_ACCESS_KEY
            value: "{{ aws_access_secret }}"
          - name: SERVER_PORT
            value: "3001"
          - name: JWT_SECRET
            value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long.
          - name: VECTOR_DB
            value: "lancedb"
          - name: STORAGE_DIR
            # value: "/data"
            value: "/app/server/storage"
          - name: NODE_ENV
            value: "development"
          - name: UID
            value: "1000"
          - name: GID
            value: "1000"
        volumeMounts: 
          - name: anything-llm-server-storage-volume-mount
            # mountPath: /data                                                                                                                                                  
            mountPath: /app/server/storage                                                                                                                                                  
      volumes:
        - name: anything-llm-server-storage-volume-mount
          persistentVolumeClaim:
            claimName: anything-llm-volume-claim

============
Symptoms: (Apologies if this is confusing. I am confused LOL)

Have dialed everything in, but getting conflicting outcomes when changing these settings:

• NODE_ENV, Values attempted: production, development
• STORAGE_DIR, Values attempted: (not set), /data, /app/server/storage

In general, when the Persistent Volume is mounted to same dir as STORAGE_DIR (/app/server/storage):

• The app does not "see" everything else in /app/server/storage.
• Mounting the persistent volume seems to mask what was there already in the image.
• We don't want to use this path in the end ... just a note for completeness.

When STORAGE_DIR is not set:

• It works perfectly, just nothing is persisted across pod restarts.
• HTTPS works perfectly.

When NODE_ENV=production, no matter the value of STORAGE_DIR:

• Site loads perfectly, nut chat workspaces do not persist.
• Can't seem to set STORAGE_DIR to anything... It keeps demanding on using /app/server/storage (IS QUITE ANNOYING).
• Confirmed that it is ignoring the setting - exec-ed into the container, and confirming contents of folders and timestamps.
• The comkey dir seems like the only directly that actually is updated in this configuration inside of the custom STORAGE_DIR.

When NODE_ENV=development & STORAGE_DIR=/data:

• Exec into the container shows the correct database has last been touched... The one in our custom directory, and not the one in the docker image. Woohoo! That means that things will persist as expected.
• There is a Not Found message when accessing the site (hosted in cluster on http, and ingress wraps it with https). I see that in /data, everything is a later timestamp that in /app/server/storage... EXCEPT with the comkey directory.

Lastly, I cannot get this thing to forget the admin setup was performed, even after clearing all data from the persistent volume and doing rollout restart.

This is super frustrating LOL, and that is a lot of info.

But, it seems like some of the build code and structural aspects of the app might not play 100% nicely with k8s.

• Its almost there, but not quite.
• In production mode, we noted that comkey is successfully saved/updated in custom STORAGE_DIR, but everything else was being updated in /app/server/storage.
• In development mode, we noted that comkey was stored in /app/server/storage while nothing was updated in STORAGE_DIR.

Anyways, I've spent 2 days on this, and I'm out of energy. Would LOVE to be able to use it.

Help @timothycarambat

• Am sure others want to run it this way.

@atljoseph commented on GitHub (May 21, 2024): We were able to make it work mostly, but something is driving me absolutely crazy. If there really is no example in the repo already, then that likely means this hasn't been done yet. In that case, I'll list the manifest that we have aas well as the symptoms. Hoping for a quick resolution, as this is a value-add all-around. ``` --- apiVersion: v1 kind: PersistentVolume metadata: name: anything-llm-volume annotations: pv.beta.kubernetes.io/uid: "1000" pv.beta.kubernetes.io/gid: "1000" spec: storageClassName: gp2 capacity: storage: 5Gi accessModes: - ReadWriteOnce awsElasticBlockStore: volumeID: ec2-ebs-volume-uuid-from-aws-console fsType: ext4 nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: topology.kubernetes.io/zone operator: In values: - us-direction-123 --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: anything-llm-volume-claim namespace: "{{ namespace }}" spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: anything-llm namespace: "{{ namespace }}" labels: anything-llm: "true" spec: selector: matchLabels: k8s-app: anything-llm replicas: 1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 0% maxUnavailable: 100% template: metadata: labels: anything-llm: "true" k8s-app: anything-llm app.kubernetes.io/name: anything-llm app.kubernetes.io/part-of: anything-llm annotations: prometheus.io/scrape: "true" prometheus.io/path: /metrics prometheus.io/port: "9090" spec: serviceAccountName: "default" terminationGracePeriodSeconds: 10 securityContext: fsGroup: 1000 runAsNonRoot: true runAsGroup: 1000 runAsUser: 1000 affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: topology.kubernetes.io/zone operator: In values: - us-direction-123 containers: - name: anything-llm resources: limits: memory: "1Gi" cpu: "500m" requests: memory: "512Mi" cpu: "250m" imagePullPolicy: IfNotPresent image: "mintplexlabs/anythingllm:master" securityContext: allowPrivilegeEscalation: true capabilities: add: - SYS_ADMIN runAsNonRoot: true runAsGroup: 1000 runAsUser: 1000 command: # Specify a command to override the Dockerfile's ENTRYPOINT. - /bin/bash - -c - | # Some debug stuff... set -x -e sleep 3 echo "AWS_REGION: $AWS_REGION" echo "SERVER_PORT: $SERVER_PORT" echo "NODE_ENV: $NODE_ENV" echo "STORAGE_DIR: $STORAGE_DIR" # The following was taken from the Dockerfile's ENTRYPOINT, since `command` overrides that. { cd /app/server/ && npx prisma generate --schema=./prisma/schema.prisma && npx prisma migrate deploy --schema=./prisma/schema.prisma && node /app/server/index.js echo "Server process exited with status $?" } & { node /app/collector/index.js echo "Collector process exited with status $?" } & wait -n exit $? readinessProbe: httpGet: path: /v1/api/health port: 8888 initialDelaySeconds: 15 periodSeconds: 5 successThreshold: 2 livenessProbe: httpGet: path: /v1/api/health port: 8888 initialDelaySeconds: 15 periodSeconds: 5 failureThreshold: 3 env: - name: AWS_REGION value: "{{ aws_region }}" - name: AWS_ACCESS_KEY_ID value: "{{ aws_access_id }}" - name: AWS_SECRET_ACCESS_KEY value: "{{ aws_access_secret }}" - name: SERVER_PORT value: "3001" - name: JWT_SECRET value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long. - name: VECTOR_DB value: "lancedb" - name: STORAGE_DIR # value: "/data" value: "/app/server/storage" - name: NODE_ENV value: "development" - name: UID value: "1000" - name: GID value: "1000" volumeMounts: - name: anything-llm-server-storage-volume-mount # mountPath: /data mountPath: /app/server/storage volumes: - name: anything-llm-server-storage-volume-mount persistentVolumeClaim: claimName: anything-llm-volume-claim ``` ============ Symptoms: (Apologies if this is confusing. I am confused LOL) ============ Have dialed everything in, but getting conflicting outcomes when changing these settings: - NODE_ENV, Values attempted: production, development - STORAGE_DIR, Values attempted: (not set), /data, /app/server/storage In general, when the Persistent Volume is mounted to same dir as STORAGE_DIR (/app/server/storage): - The app does not "see" everything else in /app/server/storage. - Mounting the persistent volume seems to mask what was there already in the image. - We don't want to use this path in the end ... just a note for completeness. When STORAGE_DIR is not set: - It works perfectly, just nothing is persisted across pod restarts. - HTTPS works perfectly. When NODE_ENV=production, no matter the value of STORAGE_DIR: - Site loads perfectly, nut chat workspaces do not persist. - Can't seem to set STORAGE_DIR to anything... It keeps demanding on using /app/server/storage (IS QUITE ANNOYING). - Confirmed that it is ignoring the setting - exec-ed into the container, and confirming contents of folders and timestamps. - The comkey dir seems like the _only_ directly that _actually_ is updated in this configuration inside of the custom STORAGE_DIR. When NODE_ENV=development & STORAGE_DIR=/data: - Exec into the container shows the correct database has last been touched... The one in our custom directory, and not the one in the docker image. **Woohoo!** That means that things will persist as expected. - There is a Not Found message when accessing the site (hosted in cluster on http, and ingress wraps it with https). I see that in /data, everything is a later timestamp that in /app/server/storage... EXCEPT with the `comkey` directory. Lastly, I cannot get this thing to forget the admin setup was performed, even after clearing all data from the persistent volume and doing rollout restart. This is super frustrating LOL, and that is a lot of info. But, it seems like some of the build code and structural aspects of the app might not play 100% nicely with k8s. - Its almost there, but not quite. - In production mode, we noted that comkey is successfully saved/updated in custom STORAGE_DIR, but everything else was being updated in /app/server/storage. - In development mode, we noted that comkey was stored in /app/server/storage while nothing was updated in STORAGE_DIR. Anyways, I've spent 2 days on this, and I'm out of energy. Would LOVE to be able to use it. Help @timothycarambat - Am sure others want to run it this way.

yindo commented 2026-02-22 18:22:16 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (May 21, 2024):

@atljoseph How do you use these templates? I don't use Kubernetes so I'm unsure how to even debug. For what it is worth, it may make sense to use modify

image: "mintplexlabs/anythingllm:master"

to image: "mintplexlabs/anythingllm:render". This image is special in that it pins all storage to have a mounted EBS attached to /storage. This is how we do persistent storage on Render and Railway because of the STORAGE_DIR flexibility that needs to exist for the Docker image, but cannot be so easily configured in ephemeral storage ecosystems (like K8) where containers spin up/down and get destroyed/created on the fly.

@timothycarambat commented on GitHub (May 21, 2024): @atljoseph How do you use these templates? I don't use Kubernetes so I'm unsure how to even debug. For what it is worth, it may make sense to use modify > image: "mintplexlabs/anythingllm:master" to `image: "mintplexlabs/anythingllm:render"`. This image is special in that it pins all storage to have a mounted EBS attached to `/storage`. This is how we do persistent storage on Render and Railway because of the STORAGE_DIR flexibility that needs to exist for the Docker image, but cannot be so easily configured in ephemeral storage ecosystems (like K8) where containers spin up/down and get destroyed/created on the fly.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

Thanks @timothycarambat I'll try that.
Curious, how do i reset everything, including the Admin user?
Doesn't seem to ever go back to the onboarding.
It would revert back to onboarding early on, but can't get it to do that anymore.
So, if you could tell me more about where all that is stored, it might help track down the issue.

@atljoseph commented on GitHub (May 21, 2024): Thanks @timothycarambat I'll try that. Curious, how do i reset everything, including the Admin user? Doesn't seem to ever go back to the onboarding. It would revert back to onboarding early on, but can't get it to do that anymore. So, if you could tell me more about where all that is stored, it might help track down the issue.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

It would be nice to be able to set Admin user through ENV Vars, or at least have the option to.

@atljoseph commented on GitHub (May 21, 2024): It would be nice to be able to set Admin user through ENV Vars, or at least have the option to.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (May 21, 2024):

Remove the LLM_PROVIDER, EMBEDDING_ENGINE and VECTOR_DB keys in the storage .env. You can also reset the user db by yarn prisma:reset in the root or in /app/server when in docker container.

You can also just rm -rf /host/storage/anythingllm.db && touch /host/storage/anythingllm.db to delete and recreate an empty db

@timothycarambat commented on GitHub (May 21, 2024): Remove the `LLM_PROVIDER`, `EMBEDDING_ENGINE` and `VECTOR_DB` keys in the storage `.env`. You can also reset the user db by `yarn prisma:reset` in the root or in `/app/server` when in docker container. You can also just `rm -rf /host/storage/anythingllm.db && touch /host/storage/anythingllm.db` to delete and recreate an empty db

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

OK, this worked grrrrreat!!!!

The render image tag worked as expected.

Final product:

---
apiVersion: v1                                                                                                                                           
kind: PersistentVolume                                                                                                                                   
metadata:                                                                                                                                                
  name: anything-llm-volume                                                                                                                              
  annotations:                                                                                                                                           
    pv.beta.kubernetes.io/uid: "1000"                                                                                                                    
    pv.beta.kubernetes.io/gid: "1000"                                                                                                                    
spec:                                                                                                                                                    
  storageClassName: gp2                                                                                                                                  
  capacity:                                                                                                                                              
    storage: 5Gi                                                                                                                                        
  accessModes:                                                                                                                                           
    - ReadWriteOnce                                                                                                                                      
  awsElasticBlockStore:    
    # This is the volumne UUID from AWS EC2 EBS Volumes list.                                                                                                                              
    volumeID: "{{ anythingllm_awsElasticBlockStore_volumeID }}"                                                                                                                           
    fsType: ext4
  nodeAffinity:                                                                                                                                          
    required:                                                                                                                                            
      nodeSelectorTerms:                                                                                                                                 
      - matchExpressions:                                                                                                                                
        - key: topology.kubernetes.io/zone                                                                                                               
          operator: In                                                                                                                                   
          values:                                                                                                                                        
          - us-east-1c  
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: anything-llm-volume-claim
  namespace: "{{ namespace }}"
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: anything-llm
  namespace: "{{ namespace }}"
  labels:
    anything-llm: "true"
spec:
  selector:
    matchLabels:
      k8s-app: anything-llm
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0%
      maxUnavailable: 100%
  template:
    metadata:
      labels:
        anything-llm: "true"
        k8s-app: anything-llm
        app.kubernetes.io/name: anything-llm
        app.kubernetes.io/part-of: anything-llm
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: /metrics
        prometheus.io/port: "9090"
    spec:
      serviceAccountName: "default"
      terminationGracePeriodSeconds: 10
      securityContext:                                                                                                                                                              
        fsGroup: 1000
        runAsNonRoot: true                                                                                                                                                          
        runAsGroup: 1000
        runAsUser: 1000
      affinity:                                                                                                                                                                                                                                                                          
        nodeAffinity:                                                                                                                                                                                                                                                                    
          requiredDuringSchedulingIgnoredDuringExecution:                                                                                                                                                                                                                                
            nodeSelectorTerms:                                                                                                                                                                                                                                                           
            - matchExpressions:                                                                                                                                                                                                                                                          
              - key: topology.kubernetes.io/zone                                                                                                                                                                                                                                         
                operator: In                                                                                                                                                                                                                                                             
                values:                                                                                                                                                                                                                                                                  
                - us-east-1c  
      containers:
      - name: anything-llm
        resources:
          limits:
            memory: "1Gi"
            cpu: "500m"
          requests:
            memory: "512Mi"
            cpu: "250m"
        imagePullPolicy: IfNotPresent
        image: "mintplexlabs/anythingllm:render"
        securityContext:                     
          allowPrivilegeEscalation: true                                                                                                                                                                                                                                                 
          capabilities:                                                                                                                                                                                                                                                                  
            add:                                                                                                                                                                                                                                                                         
              - SYS_ADMIN                                                                                                                                                                                                                                                                
          runAsNonRoot: true                                                                                                                                                                                                                                                             
          runAsGroup: 1000                                                                                                                                                                                                                                                               
          runAsUser: 1000                                                                                                                                       
        command: 
          # Specify a command to override the Dockerfile's ENTRYPOINT.
          - /bin/bash
          - -c
          - |
            set -x -e
            sleep 3
            echo "AWS_REGION: $AWS_REGION"
            echo "SERVER_PORT: $SERVER_PORT"
            echo "NODE_ENV: $NODE_ENV"
            echo "STORAGE_DIR: $STORAGE_DIR"
            {
              cd /app/server/ &&
                npx prisma generate --schema=./prisma/schema.prisma &&
                npx prisma migrate deploy --schema=./prisma/schema.prisma &&
                node /app/server/index.js
              echo "Server process exited with status $?"
            } &
            { 
              node /app/collector/index.js
              echo "Collector process exited with status $?"
            } &
            wait -n
            exit $?
        readinessProbe:
          httpGet:
            path: /v1/api/health
            port: 8888
          initialDelaySeconds: 15
          periodSeconds: 5
          successThreshold: 2
        livenessProbe:
          httpGet:
            path: /v1/api/health
            port: 8888
          initialDelaySeconds: 15
          periodSeconds: 5
          failureThreshold: 3
        env:
          - name: AWS_REGION
            value: "{{ aws_region }}"
          - name: AWS_ACCESS_KEY_ID
            value: "{{ aws_access_id }}"
          - name: AWS_SECRET_ACCESS_KEY
            value: "{{ aws_access_secret }}"
          - name: SERVER_PORT
            value: "3001"
          - name: JWT_SECRET
            value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long.
          - name: VECTOR_DB
            value: "lancedb"
          - name: STORAGE_DIR
            value: "/storage"
          - name: NODE_ENV
            value: "production"
          - name: UID
            value: "1000"
          - name: GID
            value: "1000"
        volumeMounts: 
          - name: anything-llm-server-storage-volume-mount
            mountPath: /storage                                                                                                                                                  
      volumes:
        - name: anything-llm-server-storage-volume-mount
          persistentVolumeClaim:
            claimName: anything-llm-volume-claim
---
# This serves the UI and the backend.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: anything-llm-ingress
  namespace: "{{ namespace }}"
  annotations:
    external-dns.alpha.kubernetes.io/hostname: "{{ namespace }}-chat.{{ base_domain }}"
    kubernetes.io/ingress.class: "internal-ingress"
    nginx.ingress.kubernetes.io/rewrite-target: /
    ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - host: "{{ namespace }}-chat.{{ base_domain }}"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: anything-llm-svc
            port: 
              number: 3001
  tls: # < placing a host in the TLS config will indicate a cert should be created
    - hosts:
        - "{{ namespace }}-chat.{{ base_domain }}"
      secretName: letsencrypt-prod
---
apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/name: anything-llm
  name: anything-llm-svc
  namespace: "{{ namespace }}"
spec:
  ports:
  # "port" is external port, and "targetPort" is internal.
  - port: 3301
    targetPort: 3001
    name: traffic
  - port: 9090
    targetPort: 9090
    name: metrics
  selector:
    k8s-app: anything-llm

@timothycarambat This is what is known as a kubernetes "manifest".
We don't use Kustomize, so this would have to be adapted for anyone who uses that (I don't understand it).
With Helm, we use the "render" function to apply variables that are loaded per environment.
Our Helm repo is too large just to extract a full working filesystem example here, sorry.
But, essentially... Variables are placed in handlebars, and then replaced at render time.
someK8sConfigValue: "{{ willBeReplacedAtRender }}"
So, anyone can fill in their values, and apply this directly to their cluster.
I did not include an ingress, because different folks use different strokes, depending on their needs.

Examples:

• namespace: "default" would work for someone's homelab. They'll know what they need to use if not default.
• aws variables: Well, these are required for AWS. AWS users will know and love them.
• anythingllm_awsElasticBlockStore_volumeID: This is the volume UUID from AWS EC2 EBS Volumes list.

@atljoseph commented on GitHub (May 21, 2024): OK, this worked grrrrreat!!!! The `render` image tag worked as expected. Final product: ``` --- apiVersion: v1 kind: PersistentVolume metadata: name: anything-llm-volume annotations: pv.beta.kubernetes.io/uid: "1000" pv.beta.kubernetes.io/gid: "1000" spec: storageClassName: gp2 capacity: storage: 5Gi accessModes: - ReadWriteOnce awsElasticBlockStore: # This is the volumne UUID from AWS EC2 EBS Volumes list. volumeID: "{{ anythingllm_awsElasticBlockStore_volumeID }}" fsType: ext4 nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: topology.kubernetes.io/zone operator: In values: - us-east-1c --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: anything-llm-volume-claim namespace: "{{ namespace }}" spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: anything-llm namespace: "{{ namespace }}" labels: anything-llm: "true" spec: selector: matchLabels: k8s-app: anything-llm replicas: 1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 0% maxUnavailable: 100% template: metadata: labels: anything-llm: "true" k8s-app: anything-llm app.kubernetes.io/name: anything-llm app.kubernetes.io/part-of: anything-llm annotations: prometheus.io/scrape: "true" prometheus.io/path: /metrics prometheus.io/port: "9090" spec: serviceAccountName: "default" terminationGracePeriodSeconds: 10 securityContext: fsGroup: 1000 runAsNonRoot: true runAsGroup: 1000 runAsUser: 1000 affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: topology.kubernetes.io/zone operator: In values: - us-east-1c containers: - name: anything-llm resources: limits: memory: "1Gi" cpu: "500m" requests: memory: "512Mi" cpu: "250m" imagePullPolicy: IfNotPresent image: "mintplexlabs/anythingllm:render" securityContext: allowPrivilegeEscalation: true capabilities: add: - SYS_ADMIN runAsNonRoot: true runAsGroup: 1000 runAsUser: 1000 command: # Specify a command to override the Dockerfile's ENTRYPOINT. - /bin/bash - -c - | set -x -e sleep 3 echo "AWS_REGION: $AWS_REGION" echo "SERVER_PORT: $SERVER_PORT" echo "NODE_ENV: $NODE_ENV" echo "STORAGE_DIR: $STORAGE_DIR" { cd /app/server/ && npx prisma generate --schema=./prisma/schema.prisma && npx prisma migrate deploy --schema=./prisma/schema.prisma && node /app/server/index.js echo "Server process exited with status $?" } & { node /app/collector/index.js echo "Collector process exited with status $?" } & wait -n exit $? readinessProbe: httpGet: path: /v1/api/health port: 8888 initialDelaySeconds: 15 periodSeconds: 5 successThreshold: 2 livenessProbe: httpGet: path: /v1/api/health port: 8888 initialDelaySeconds: 15 periodSeconds: 5 failureThreshold: 3 env: - name: AWS_REGION value: "{{ aws_region }}" - name: AWS_ACCESS_KEY_ID value: "{{ aws_access_id }}" - name: AWS_SECRET_ACCESS_KEY value: "{{ aws_access_secret }}" - name: SERVER_PORT value: "3001" - name: JWT_SECRET value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long. - name: VECTOR_DB value: "lancedb" - name: STORAGE_DIR value: "/storage" - name: NODE_ENV value: "production" - name: UID value: "1000" - name: GID value: "1000" volumeMounts: - name: anything-llm-server-storage-volume-mount mountPath: /storage volumes: - name: anything-llm-server-storage-volume-mount persistentVolumeClaim: claimName: anything-llm-volume-claim --- # This serves the UI and the backend. apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: anything-llm-ingress namespace: "{{ namespace }}" annotations: external-dns.alpha.kubernetes.io/hostname: "{{ namespace }}-chat.{{ base_domain }}" kubernetes.io/ingress.class: "internal-ingress" nginx.ingress.kubernetes.io/rewrite-target: / ingress.kubernetes.io/ssl-redirect: "false" spec: rules: - host: "{{ namespace }}-chat.{{ base_domain }}" http: paths: - path: / pathType: Prefix backend: service: name: anything-llm-svc port: number: 3001 tls: # < placing a host in the TLS config will indicate a cert should be created - hosts: - "{{ namespace }}-chat.{{ base_domain }}" secretName: letsencrypt-prod --- apiVersion: v1 kind: Service metadata: labels: kubernetes.io/name: anything-llm name: anything-llm-svc namespace: "{{ namespace }}" spec: ports: # "port" is external port, and "targetPort" is internal. - port: 3301 targetPort: 3001 name: traffic - port: 9090 targetPort: 9090 name: metrics selector: k8s-app: anything-llm ``` @timothycarambat This is what is known as a kubernetes "manifest". We don't use Kustomize, so this would have to be adapted for anyone who uses that (I don't understand it). With Helm, we use the "render" function to apply variables that are loaded per environment. Our Helm repo is too large just to extract a full working filesystem example here, sorry. But, essentially... Variables are placed in handlebars, and then replaced at render time. `someK8sConfigValue: "{{ willBeReplacedAtRender }}"` So, anyone can fill in their values, and apply this directly to their cluster. I did not include an ingress, because different folks use different strokes, depending on their needs. Examples: - namespace: "default" would work for someone's homelab. They'll know what they need to use if not default. - aws variables: Well, these are required for AWS. AWS users will know and love them. - anythingllm_awsElasticBlockStore_volumeID: This is the volume UUID from AWS EC2 EBS Volumes list.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

Hoping this helps someone else !

@atljoseph commented on GitHub (May 21, 2024): Hoping this helps someone else !

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

Getting an error and empty scrren when visiting LLM Preferences from Settings.

@atljoseph commented on GitHub (May 21, 2024): Getting an error and empty scrren when visiting LLM Preferences from Settings. <img width="1718" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/25017621/aeb517b3-1eec-4a0b-a308-e9864505e97a"> <img width="1188" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/25017621/3b09b137-9b23-484b-8a56-688a4cc11e67">

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

All other screens function. Not ure why this one wouldn't. Maybe because it is the one I need LOL.

@atljoseph commented on GitHub (May 21, 2024): All other screens function. Not ure why this one wouldn't. Maybe because it is the one I need LOL. <img width="977" alt="image" src="https://github.com/Mintplex-Labs/anything-llm/assets/25017621/be461f92-b34d-4e33-8070-c6341d89392f">

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

I think this needs a way for Admin to reset everything from the frontend.

@atljoseph commented on GitHub (May 21, 2024): I think this needs a way for Admin to reset everything from the frontend.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

You can also just rm -rf /host/storage/anythingllm.db && touch /host/storage/anythingllm.db to delete and recreate an empty db

Did that... All the chat workspaces were cleared out...

• But on reload was still signed in as the admin user.

Does that mean the users are not stored in anythingllm.db?

• I'm assuming this is just SQLite, right?

Remove the LLM_PROVIDER, EMBEDDING_ENGINE and VECTOR_DB keys in the storage .env.

None of these are set currently. At deploy/config time, we load ENV Vars from key-value store, and then apply the changes to kubernetes. We don't have an ENV file. Kubernetes supplies these, and if they are modified, they do not persist across restarts.

You can also reset the user db by yarn prisma:reset in the root or in /app/server when in docker container.

Can you tell me more about that, please? Where is this prisma DB stored? Looking to have a solution that does not involve exec-ing to an ephemeral container.

@atljoseph commented on GitHub (May 21, 2024): > You can also just rm -rf /host/storage/anythingllm.db && touch /host/storage/anythingllm.db to delete and recreate an empty db Did that... All the chat workspaces were cleared out... - But on reload was still signed in as the admin user. Does that mean the users are not stored in anythingllm.db? - I'm assuming this is just SQLite, right? > Remove the LLM_PROVIDER, EMBEDDING_ENGINE and VECTOR_DB keys in the storage .env. None of these are set currently. At deploy/config time, we load ENV Vars from key-value store, and then apply the changes to kubernetes. We don't have an ENV file. Kubernetes supplies these, and if they are modified, they do not persist across restarts. > You can also reset the user db by yarn prisma:reset in the root or in /app/server when in docker container. Can you tell me more about that, please? Where is this prisma DB stored? Looking to have a solution that does not involve exec-ing to an ephemeral container.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (May 21, 2024):

@atljoseph that ENV error is because in

 env:
          - name: AWS_REGION
            value: "{{ aws_region }}"
          - name: AWS_ACCESS_KEY_ID
            value: "{{ aws_access_id }}"
          - name: AWS_SECRET_ACCESS_KEY
            value: "{{ aws_access_secret }}"
          - name: SERVER_PORT
            value: "3001"
          - name: JWT_SECRET
            value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long.
          - name: VECTOR_DB
            value: "lancedb"
          - name: STORAGE_DIR
            value: "/storage"
          - name: NODE_ENV
            value: "production"
          - name: UID
            value: "1000"
          - name: GID
            value: "1000"

You have

- name: VECTOR_DB
  value: "lancedb"

Remove this key and you can onboard and that LLM preference bug wont appear. Onboarding set these vars

@timothycarambat commented on GitHub (May 21, 2024): @atljoseph that ENV error is because in ``` env: - name: AWS_REGION value: "{{ aws_region }}" - name: AWS_ACCESS_KEY_ID value: "{{ aws_access_id }}" - name: AWS_SECRET_ACCESS_KEY value: "{{ aws_access_secret }}" - name: SERVER_PORT value: "3001" - name: JWT_SECRET value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long. - name: VECTOR_DB value: "lancedb" - name: STORAGE_DIR value: "/storage" - name: NODE_ENV value: "production" - name: UID value: "1000" - name: GID value: "1000" ``` You have ``` - name: VECTOR_DB value: "lancedb" ``` Remove this key and you can onboard and that LLM preference bug wont appear. Onboarding set these vars

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 21, 2024):

Thank you sir. Will try that tomorrow!

On Tue, May 21, 2024 at 10:34 PM Timothy Carambat @.***>
wrote:

@atljoseph https://github.com/atljoseph that ENV error is because in

env:
- name: AWS_REGION
value: "{{ aws_region }}"
- name: AWS_ACCESS_KEY_ID
value: "{{ aws_access_id }}"
- name: AWS_SECRET_ACCESS_KEY
value: "{{ aws_access_secret }}"
- name: SERVER_PORT
value: "3001"
- name: JWT_SECRET
value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long.
- name: VECTOR_DB
value: "lancedb"
- name: STORAGE_DIR
value: "/storage"
- name: NODE_ENV
value: "production"
- name: UID
value: "1000"
- name: GID
value: "1000"

You have

• name: VECTOR_DB
  value: "lancedb"

Remove this key and you can onboard and that LLM preference bug wont
appear. Onboarding set these vars

—
Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/1463#issuecomment-2123759591,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AF632FLECBYSW2GIK6ODZHTZDP73ZAVCNFSM6AAAAABH7XQKCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRTG42TSNJZGE
.
You are receiving this because you were mentioned.Message ID:
@.***>

@atljoseph commented on GitHub (May 21, 2024): Thank you sir. Will try that tomorrow! On Tue, May 21, 2024 at 10:34 PM Timothy Carambat ***@***.***> wrote: > @atljoseph <https://github.com/atljoseph> that ENV error is because in > > env: > - name: AWS_REGION > value: "{{ aws_region }}" > - name: AWS_ACCESS_KEY_ID > value: "{{ aws_access_id }}" > - name: AWS_SECRET_ACCESS_KEY > value: "{{ aws_access_secret }}" > - name: SERVER_PORT > value: "3001" > - name: JWT_SECRET > value: "my-random-string-for-seeding" # Please generate random string at least 12 chars long. > - name: VECTOR_DB > value: "lancedb" > - name: STORAGE_DIR > value: "/storage" > - name: NODE_ENV > value: "production" > - name: UID > value: "1000" > - name: GID > value: "1000" > > You have > > - name: VECTOR_DB > value: "lancedb" > > Remove this key and you can onboard and that LLM preference bug wont > appear. Onboarding set these vars > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/1463#issuecomment-2123759591>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AF632FLECBYSW2GIK6ODZHTZDP73ZAVCNFSM6AAAAABH7XQKCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRTG42TSNJZGE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (May 23, 2024):

Worked. Everything above in my last message with k8s manifest (minus VECTOR_DB) worked as expected, especially after removing that env var.

@atljoseph commented on GitHub (May 23, 2024): Worked. Everything above in my last message with k8s manifest (minus VECTOR_DB) worked as expected, especially after removing that env var.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@reefland commented on GitHub (Jun 23, 2024):

@atljoseph - In your templates readinessProbe and livenessProbe are using port 8888, and while the logs report Document processor app listening on port 8888 I'm unable to get curl to respond to that port, I have to CTRL-C to stop.:

[ root@curl:/ ]$ curl -v anything-llm.anything-llm:8888/v1/api/health
^C

I don't see how your probes could be successful. I instead use probes against port 3001:

[ root@curl:/ ]$ curl anything-llm.anything-llm:3001
<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8" />
  <link rel="icon" type="image/svg+xml" href="/favicon.png" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>AnythingLLM | Your personal LLM trained on anything</title>
...

And path doesn't matter. Seems to get same output no matter what you ask for.

[ root@curl:/ ]$ curl anything-llm.anything-llm:3001/blah/blah/blah
<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8" />
  <link rel="icon" type="image/svg+xml" href="/favicon.png" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>AnythingLLM | Your personal LLM trained on anything</title>
...

In regards to Prometheus metrics, you have annotations for scrapes to port 9090 and setup that port for the service, but there is nothing within the Anything-LLM container listening to that port that I could find. If it doesn't exist that could be cleaned up.

I'd be curious to know why you needed to grant allowPrivilegeEscalation: true and grant SYS_ADMIN capabilities to the container:

       securityContext:                     
          allowPrivilegeEscalation: true                                                                                                                                                                                                                                                 
          capabilities:                                                                                                                                                                                                                                                                  
            add:                                                                                                                                                                                                                                                                         
              - SYS_ADMIN                                                                                                                                                                                                                                                                
          runAsNonRoot: true                                                                                                                                                                                                                                                             
          runAsGroup: 1000                                                                                                                                                                                                                                                               
          runAsUser: 1000

I find Anything-LLM runs OK, locked down a bit more at the container level:

      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop:
            - ALL
        readOnlyRootFilesystem: false

Then at the Pod level you can do:

  securityContext:
    fsGroup: 1000
    runAsGroup: 1000
    runAsNonRoot: true
    runAsUser: 1000

For others trying this. The .env file concept does not work well with Kubernetes. In Kubernetes this would be stored in a configMap and either mounted into the container's filesystem or the configMap values mapped to ENV variables. Being that some of the values are sensitive then these would be stored in a secret and mapped to ENV variables.

Trying to mount the .env file from a configMap works when NODE_ENV='production' is used, unfortunately Anything-llm tries to update its .env file in development mode which breaks and crashes as configMap is read-only. Otherwise you have to do some silly trickery with an InitContainer where you could mount the .env file to something like /tmp and then use commands manually copy it to /app/server/ directory to make it writeable (but changes not persisted) before the anything-llm container is started.

@reefland commented on GitHub (Jun 23, 2024): @atljoseph - In your templates `readinessProbe` and `livenessProbe` are using port `8888`, and while the logs report `Document processor app listening on port 8888` I'm unable to get curl to respond to that port, I have to CTRL-C to stop.: ``` [ root@curl:/ ]$ curl -v anything-llm.anything-llm:8888/v1/api/health ^C ``` I don't see how your probes could be successful. I instead use probes against port 3001: ``` [ root@curl:/ ]$ curl anything-llm.anything-llm:3001 <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8" /> <link rel="icon" type="image/svg+xml" href="/favicon.png" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>AnythingLLM | Your personal LLM trained on anything</title> ... ``` And path doesn't matter. Seems to get same output no matter what you ask for. ``` [ root@curl:/ ]$ curl anything-llm.anything-llm:3001/blah/blah/blah <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8" /> <link rel="icon" type="image/svg+xml" href="/favicon.png" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>AnythingLLM | Your personal LLM trained on anything</title> ... ``` In regards to Prometheus metrics, you have annotations for scrapes to port 9090 and setup that port for the service, but there is nothing within the Anything-LLM container listening to that port that I could find. If it doesn't exist that could be cleaned up. I'd be curious to know why you needed to grant `allowPrivilegeEscalation: true` and grant SYS_ADMIN capabilities to the container: ``` securityContext: allowPrivilegeEscalation: true capabilities: add: - SYS_ADMIN runAsNonRoot: true runAsGroup: 1000 runAsUser: 1000 ``` I find Anything-LLM runs OK, locked down a bit more at the container level: ``` securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: false ``` Then at the Pod level you can do: ``` securityContext: fsGroup: 1000 runAsGroup: 1000 runAsNonRoot: true runAsUser: 1000 ``` For others trying this. The `.env` file concept does not work well with Kubernetes. In Kubernetes this would be stored in a configMap and either mounted into the container's filesystem or the configMap values mapped to ENV variables. Being that some of the values are sensitive then these would be stored in a secret and mapped to ENV variables. Trying to mount the `.env` file from a configMap works when `NODE_ENV='production'` is used, unfortunately Anything-llm tries to update its `.env` file in development mode which breaks and crashes as configMap is read-only. Otherwise you have to do some silly trickery with an InitContainer where you could mount the `.env` file to something like `/tmp` and then use commands manually copy it to `/app/server/` directory to make it writeable (but changes not persisted) before the anything-llm container is started.

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (Jun 23, 2024):

@reefland some commentary

In your templates readinessProbe and livenessProbe are using port 8888, and while the logs report Document processor app listening on port 8888 I'm unable to get curl to respond to that port, I have to CTRL-C to stop.:

This is normal in the docker container. The 8888 port is not supposed to be exposed however, it is your docker container so you can if you like. The default is to now because there is really no reason to have that be the case for the collector currently.

And path doesn't matter. Seems to get same output no matter what you ask for.

In production build/docker the root path of 3001 runs the frontend and 3001/api runs all backend endpoints for the app. Getting a response on any path is the HTML content being returns from an unmatched path while /api/ping indicates the backend server endpoints are running. In either case, if the backend was not online - not even the frontend would return a response.

I'd be curious to know why you needed to grant allowPrivilegeEscalation: true and grant SYS_ADMIN capabilities to the container:

This is in the official image for our Docker container. You cannot run a sandbox'd chromium browser without this capability on the base image we have unless you want to run the browser without sandboxing. Obviously, we want sandboxing - so that capability must exist. This is used for scraping websites so that SPA/JS apps can render first and we can scrape that.

As for the other comments, seems to be use case specific settings or tooling so I have nothing to add around that

@timothycarambat commented on GitHub (Jun 23, 2024): @reefland some commentary > In your templates readinessProbe and livenessProbe are using port 8888, and while the logs report Document processor app listening on port 8888 I'm unable to get curl to respond to that port, I have to CTRL-C to stop.: This is normal in the docker container. The 8888 port is not _supposed_ to be exposed however, it is your docker container so you can if you like. The default is to now because there is really no reason to have that be the case for the collector currently. > And path doesn't matter. Seems to get same output no matter what you ask for. In production build/docker the root path of 3001 runs the frontend and 3001/api runs all backend endpoints for the app. Getting a response on any path is the HTML content being returns from an unmatched path while /api/ping indicates the backend server endpoints are running. In either case, if the backend was not online - not even the frontend would return a response. > I'd be curious to know why you needed to grant allowPrivilegeEscalation: true and grant SYS_ADMIN capabilities to the container: This is in the official image for our Docker container. You cannot run a sandbox'd chromium browser without this capability on the base image we have unless you want to run the browser without sandboxing. Obviously, we want sandboxing - so that capability must exist. This is used for scraping websites so that SPA/JS apps can render first and we can scrape that. As for the other comments, seems to be use case specific settings or tooling so I have nothing to add around that

yindo commented 2026-02-22 18:22:17 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (Jun 23, 2024):

Yo. It’s working fine for me.

On Sun, Jun 23, 2024 at 6:05 PM Timothy Carambat @.***>
wrote:

@reefland https://github.com/reefland some commentary

In your templates readinessProbe and livenessProbe are using port 8888,
and while the logs report Document processor app listening on port 8888 I'm
unable to get curl to respond to that port, I have to CTRL-C to stop.:

This is normal in the docker container. The 8888 port is not supposed
to be exposed however, it is your docker container so you can if you like.
The default is to now because there is really no reason to have that be the
case for the collector currently.

And path doesn't matter. Seems to get same output no matter what you ask
for.

In production build/docker the root path of 3001 runs the frontend and
3001/api runs all backend endpoints for the app. Getting a response on any
path is the HTML content being returns from an unmatched path while
/api/ping indicates the backend server endpoints are running. In either
case, if the backend was not online - not even the frontend would return a
response.

I'd be curious to know why you needed to grant allowPrivilegeEscalation:
true and grant SYS_ADMIN capabilities to the container:

This is in the official image for our Docker container. You cannot run a
sandbox'd chromium browser without this capability on the base image we
have unless you want to run the browser without sandboxing. Obviously, we
want sandboxing - so that capability must exist. This is used for scraping
websites so that SPA/JS apps can render first and we can scrape that.

As for the other comments, seems to be use case specific settings or
tooling so I have nothing to add around that

—
Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/1463#issuecomment-2185337089,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AF632FPTHIKEV3MKKPPUGHDZI5BBPAVCNFSM6AAAAABH7XQKCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGMZTOMBYHE
.
You are receiving this because you were mentioned.Message ID:
@.***>

@atljoseph commented on GitHub (Jun 23, 2024): Yo. It’s working fine for me. On Sun, Jun 23, 2024 at 6:05 PM Timothy Carambat ***@***.***> wrote: > @reefland <https://github.com/reefland> some commentary > > In your templates readinessProbe and livenessProbe are using port 8888, > and while the logs report Document processor app listening on port 8888 I'm > unable to get curl to respond to that port, I have to CTRL-C to stop.: > > This is normal in the docker container. The 8888 port is not *supposed* > to be exposed however, it is your docker container so you can if you like. > The default is to now because there is really no reason to have that be the > case for the collector currently. > > And path doesn't matter. Seems to get same output no matter what you ask > for. > > In production build/docker the root path of 3001 runs the frontend and > 3001/api runs all backend endpoints for the app. Getting a response on any > path is the HTML content being returns from an unmatched path while > /api/ping indicates the backend server endpoints are running. In either > case, if the backend was not online - not even the frontend would return a > response. > > I'd be curious to know why you needed to grant allowPrivilegeEscalation: > true and grant SYS_ADMIN capabilities to the container: > > This is in the official image for our Docker container. You cannot run a > sandbox'd chromium browser without this capability on the base image we > have unless you want to run the browser without sandboxing. Obviously, we > want sandboxing - so that capability must exist. This is used for scraping > websites so that SPA/JS apps can render first and we can scrape that. > > As for the other comments, seems to be use case specific settings or > tooling so I have nothing to add around that > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/1463#issuecomment-2185337089>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AF632FPTHIKEV3MKKPPUGHDZI5BBPAVCNFSM6AAAAABH7XQKCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGMZTOMBYHE> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@stguitar commented on GitHub (Jun 25, 2024):

would it make more sense for folks to have the committed/provided sample k8s manifests to work with a standard production build docker image?

• have the liveness and readiness probes use /api/ping or /api/utils/metrics on port 3001
• drop the metrics and healthcheck config as they are ports that are not exposed by the container by default (the current setup should result in a pod that won't start)

I get that its somewhat a personal preference, but it seems that it would help to have the default k8s behavior align with the default docker setup.

Further, while these env settings might work, it would be more straight forward to just have an additional example resource of a configmap to configure those. then that would be directly applicable as a starting point, and folks could swap to secrets instead of they so choose.

I don't say all this without the offer of a PR, but I wanted to make sure before I do that, that it might be something that would be merged.

@stguitar commented on GitHub (Jun 25, 2024): would it make more sense for folks to have the committed/provided sample k8s manifests to work with a standard production build docker image? * have the liveness and readiness probes use `/api/ping` or `/api/utils/metrics` on port `3001` * drop the metrics and healthcheck config as they are ports that are not exposed by the container by default (the current setup should result in a pod that won't start) I get that its somewhat a personal preference, but it seems that it would help to have the default k8s behavior align with the default docker setup. Further, while these env settings might work, it would be more straight forward to just have an additional example resource of a `configmap` to configure those. then that would be directly applicable as a starting point, and folks could swap to secrets instead of they so choose. I don't say all this without the offer of a PR, but I wanted to make sure before I do that, that it might be something that would be merged.

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (Jun 26, 2024):

@stguitar if you can produce a manifest/template for K8s that:

• works as-is on deployment (ENV setting aside, of course)
• Aligned with the intended functionality and use of the regular docker container

then I will merge it in for the cloud-deployments folder, no question!

If there are supplemental "gotchas" or key pieces a reasonable person may overlook and cannot be commented into the template I am happy to make a page and document those details on our documentation page as well.

@timothycarambat commented on GitHub (Jun 26, 2024): @stguitar if you can produce a manifest/template for K8s that: - works as-is on deployment (ENV setting aside, of course) - Aligned with the intended functionality and use of the regular docker container then I will merge it in for the `cloud-deployments` folder, no question! If there are supplemental "gotchas" or key pieces a reasonable person may overlook and cannot be commented into the template I am happy to make a page and document those details on our [documentation page as well.](https://github.com/Mintplex-Labs/anythingllm-docs)

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (Jun 26, 2024):

Sorry, I didn’t really intend to provide a perfect file others could exactly copy, but one which held the required ideas, and actually functions for a given cluster. Of course some things could be improved. Every cluster is different. And some use this or that tool to configure things. The key points def include container permissions, env vars, volume persistence, and … use of the “render” image. Many hours were spent here learning that lesson the hard way.

@atljoseph commented on GitHub (Jun 26, 2024): Sorry, I didn’t really intend to provide a perfect file others could exactly copy, but one which held the required ideas, and actually functions for a given cluster. Of course some things could be improved. Every cluster is different. And some use this or that tool to configure things. The key points def include container permissions, env vars, volume persistence, and … use of the “render” image. Many hours were spent here learning that lesson the hard way.

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (Jun 26, 2024):

@atljoseph understandable and I was on the same page as you that this was your specific endeavor and use-case and it may not be portable to others.

The PR still stands though if someone does want to contribute a general-use K8 template

@timothycarambat commented on GitHub (Jun 26, 2024): @atljoseph understandable and I was on the same page as you that this was your specific endeavor and use-case and it may not be portable to others. The PR still stands though if someone does want to contribute a general-use K8 template

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@stguitar commented on GitHub (Jun 26, 2024):

Good to know guys... thank you for the replies and the effort on all this previously. I definitely didn't mean to not be thankful for what you have provided in this PR. I was curious - can either of you expand on the "render" image and what it's configuration is? Maybe it opens ports that the normal image doesn't.

I guess how it felt to me, and im presuming the same for @reefland was that if these files don't actually work at all (which they won't unless the "render" container tag exposes ports that the standard port does not), it seems plausible to just have text docs that list out the suggested resources one might need to deploy, and point folks to the standard (or even random) examples of those resources from the web. You could further call out the "special sauce" (like the shell scripting, and the pod's security context stuff).

I totally agree that people have different deployment needs. Heck, I won't be using AWS. However, these manifests seem to suggest that the ports of 9090 and 8888 are exposed, which I don't think they are. They aren't in the standard image anyway. Maybe they are in the render image? It just felt like the interaction (best word I can think of) of the pod and the container could be a bit tighter. Not so much the rest of the "environmental selection of the day" stuff, like env vars, secrets, config maps, ingresses (which i also don't happen to need). I did mention the latter stuff, but that wasn't as big of a gap for me personally as much as the pod/container cohesion.

That being said, I could be totally wrong and the "render" container actually exposes those ports and all is well!

Depending on your thoughts and at this point from my point of view, I am still testing out my version of the k8s manifests and should that go well, I would be happy to share what I have in a PR for consideration.

@stguitar commented on GitHub (Jun 26, 2024): Good to know guys... thank you for the replies and the effort on all this previously. I definitely didn't mean to not be thankful for what you have provided in this PR. I was curious - can either of you expand on the "render" image and what it's configuration is? Maybe it opens ports that the normal image doesn't. I guess how it felt to me, and im presuming the same for @reefland was that if these files don't actually work at all (which they won't unless the "render" container tag exposes ports that the standard port does not), it seems plausible to just have text docs that list out the suggested resources one might need to deploy, and point folks to the standard (or even random) examples of those resources from the web. You could further call out the "special sauce" (like the shell scripting, and the pod's security context stuff). I totally agree that people have different deployment needs. Heck, I won't be using AWS. However, these manifests seem to suggest that the ports of 9090 and 8888 are exposed, which I don't think they are. They aren't in the standard image anyway. Maybe they are in the render image? It just felt like the interaction (best word I can think of) of the `pod` and the `container` could be a bit tighter. Not so much the rest of the "environmental selection of the day" stuff, like env vars, secrets, config maps, ingresses (which i also don't happen to need). I did mention the latter stuff, but that wasn't as big of a gap for me personally as much as the pod/container cohesion. That being said, I could be totally wrong and the "render" container actually exposes those ports and all is well! Depending on your thoughts and at this point from my point of view, I am still testing out my version of the k8s manifests and should that go well, I would be happy to share what I have in a PR for consideration.

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@timothycarambat commented on GitHub (Jun 26, 2024):

The only public port that should be exposed in the docker container running AnythingLLM in any template is 3001.
Def dont expose 8888 and looking at the current K8 community template 9090 seems to be use-case specific so we can likely omit that as well

I dont specifically use K8's - which is why I lean on the community to publish or amend that template. I truthfully have no experience in that area.

@timothycarambat commented on GitHub (Jun 26, 2024): The only public port that should be exposed in the docker container running AnythingLLM in any template is 3001. Def dont expose 8888 and looking at the current K8 community template 9090 seems to be use-case specific so we can likely omit that as well I dont specifically use K8's - which is why I lean on the community to publish or amend that template. I truthfully have no experience in that area.

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@atljoseph commented on GitHub (Jun 26, 2024):

Don’t get confused by those healthchecks on port 8888. Those can simply be
removed if you don’t utilize them. 9090 is metrics. Again, if you don’t
have a need, then just remove them. These are all somewhat standard k8s
concepts.

I haven’t been able to put my finger on why the render image works for k8s
and the other doesn’t. Feels like it should just be possible to utilize the
functional result that either image provides, but with just one image
instead, and relying on env vars to affect configuration.

We also ended up going with a non-standard port in the end, like 3002,
since 3001 is a standard node port and conflicted with another service.
Works fine though.

On Wed, Jun 26, 2024 at 8:23 PM Timothy Carambat @.***>
wrote:

The only public port that should be exposed in the docker container
running AnythingLLM in any template is 3001.
Def dont expose 8888 and looking at the current K8 community template 9090
seems to be use-case specific so we can likely omit that as well

—
Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/1463#issuecomment-2192832048,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AF632FONE4H45CFE4T7SZHDZJNLPPAVCNFSM6AAAAABH7XQKCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSHAZTEMBUHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

@atljoseph commented on GitHub (Jun 26, 2024): Don’t get confused by those healthchecks on port 8888. Those can simply be removed if you don’t utilize them. 9090 is metrics. Again, if you don’t have a need, then just remove them. These are all somewhat standard k8s concepts. I haven’t been able to put my finger on why the render image works for k8s and the other doesn’t. Feels like it should just be possible to utilize the functional result that either image provides, but with just one image instead, and relying on env vars to affect configuration. We also ended up going with a non-standard port in the end, like 3002, since 3001 is a standard node port and conflicted with another service. Works fine though. On Wed, Jun 26, 2024 at 8:23 PM Timothy Carambat ***@***.***> wrote: > The only public port that should be exposed in the docker container > running AnythingLLM in any template is 3001. > Def dont expose 8888 and looking at the current K8 community template 9090 > seems to be use-case specific so we can likely omit that as well > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/1463#issuecomment-2192832048>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AF632FONE4H45CFE4T7SZHDZJNLPPAVCNFSM6AAAAABH7XQKCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSHAZTEMBUHA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

yindo commented 2026-02-22 18:22:18 -05:00

Author

Owner

Copy Link

Copy Source

@reefland commented on GitHub (Jun 26, 2024):

The only public port that should be exposed in the docker container running AnythingLLM in any template is 3001. Def dont expose 8888 and looking at the current K8 community template 9090 seems to be use-case specific so we can likely omit that as well

Unless you include a Prometheus Exporter that listens on port 9090 as part of your image, then this port and respective annotations should be removed. It's not use-case specific if it simply does not exist.

@reefland commented on GitHub (Jun 26, 2024): > The only public port that should be exposed in the docker container running AnythingLLM in any template is 3001. Def dont expose 8888 and looking at the current K8 community template 9090 seems to be use-case specific so we can likely omit that as well > Unless you include a Prometheus Exporter that listens on port 9090 as part of your image, then this port and respective annotations should be removed. It's not use-case specific if it simply does not exist.

yindo referenced this issue 2026-06-05 14:36:00 -04:00

[GH-ISSUE #934] always return answer without context file #562

yindo changed title from [DOCS]: Basic k8s kubernetes manifest. to [GH-ISSUE #1463] [DOCS]: Basic k8s kubernetes manifest. 2026-06-05 14:38:02 -04:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

master

5846-bug-when-scrolling-up-scrolling-jumps

refactor-remove-workspace-pfp

5969-bug-stopgenerationbutton-disappears-on-the-first-prompt-that-initiates-an-agent-session

2235-bug-how-to-upload-a-folder-with-subfolders-with-files-to-anythingllm

5990-workspace-update-fails-with-unknown-argument-router_id-v1130v1150-intel-mac

opencomputer-examples

pg

feat/image-generation-translations

feat/image-generation

5924-bug-meeting-summary-fails-with-sincludes-is-not-a-function-when-default-llm-is-anthropic-claude

render

feat/uniform-modal-component

5883-bug-unescaped-content-in-json-strings-being-passed-to-document-generator-tools

5901-bug-api-update-embeddings-fails-prisma-argument-filename-is-missing-on-workspace_documentscreate-desktop-windows-v1141

hybrid-search

1981-translations

5752-bug-prompts-to-local-jan-endpoint-unresponsive

feat-disable-native-tool-calling-env-var

5676-bug-non-ollama-agent-providers-do-not-parse-and-present-reasoning-content

feat/markdown-web-scraping

5717-bug-apiv1documentupload-silently-drops-metadata-field-in-desktop-1130-arg-count-mismatch-nested-payload-key

fix/aibitat-context-overflow

5711-bug-erratic-deepseek-v4-flash-the-agent-model-failed-to-respond-400-the-reasoning_content

5631-feat-custom-api-request-timeouts-for-ai-providers

feat-reasoning-control

feat-agent-clarifying-questions-translations

5583-bug-lm-studio-provider-does-not-present-reasoning-output

5313-normalize-translations

feat/memory-translations

5305-lemonade-embedding-engine-swallows-errors-falsely-reports-documents-as-embedded

5060-bug-agent-interactions-agent-are-not-persisted-to-thread-history-via-api

pptx-subagent

feat-render-images-from-mcp-tool-results

stt-provider-expansion-openai-api-compatible

feat-file-search-agent-tool

feat-native-embedder-job-queue

5189-normalize-translations

feat-file-search-agent-tool-translations

i18n-eslint

5140-auto-migration

5112-bug-openrouter-failed-message-bug

3506-feat-parameters-for-openrouter-models

4992-feat-preserve-scroll-position

4973-bug-markdown-numbered-list-display-in-reasoning-pane

desktop

4938-bug-pending-chat-rerendering-ui-bug

quickstart-env

node-llama-cpp-in-container-cuda

node-llama-cpp-in-container

ollama-in-container

4817-feat-set-cooldown-per-mcp-server

4845-keyboard-shortcuts-to-navigate-in-chat

4844-feat-reorder-threads-by-latest-interaction

standardize-username-constraints-normalize-translations

4792-feat-refactor-workspacepfp-image

1382-embed-ip-improvements

1382-bug-embed-api-improvements

refactor-eslint-frontend

4687-feat-refactor-vector-db-providers

4615-feat-disable-apidocs-with-environment-variable

4559-feat-agent-web-search-enable-ordering-of-results

4599-bug-ollama-race-condition-bug

4572-bug-lmstudio-provided-llm-stopped-working-with-anything-llm-after-upgrading-to-190

4508-agent-youtube-transcript-analysis

4497-feat-workspace-names

frontend-eslint

ollama-lmstudio-auto-context-window

4431-validate-vector-database-connectioN

2019-slash-command-keyboard-selection

microsoft-foundry-provider

4431-validate-vector-database-connection

4325-sys-prompt-var-improvements

3209-feat-apiv1workspacestream-chat-sources-citations

4210-bug-voice-to-text-overwrite

4136-feat-jan-as-a-backend-server-option

4172-feat-openai-o3-support

1.8.3-rerelease

web-push-notifications-service

tasks

3955-feat-jinaai-embedder-provider-support

3921-feat-agent-skills-uiux-improvements

3901-bug-validfunccall-checks-optional-arguments

keyboard-dev

1787-custom-roles-and-permissions

add-jira-slack-data-connector

office-extension-wip

lightmode-dropdown-color-update

3586-bug-agent-flow-function-description-provided-by-user-is-not-seen-in-the-llm-query

3463-bug-agent-continues-to-run-if-request-failed-even-after-exit

3439-feat-call-variables-within-the-flow-api-block-url-field

3282-manager-view-models-workspace

3280-token-counting-server-side-truncation-improvements

3147-bug-embedded-chat-widget---not-considering-query-mode-option-always-working-in-chat-mode

2995-feat-disable-temperature-setting-for-deepseek-r1-deepseek-reasoner-model

2827-feat-perplexity-citations

2866-feat-finally-a-gemini-models-endpoint

2647-feat-hpp-header-for-a-c++-code-file-mime-addition

lancedb-revert

1656-feat-implement-tooltip-ui-designs

2011-feat-bump-perplexity-models

1873-feat-auto-add-and-watch-folder-for-document-uploads

1297-feat-gemini-agent-support

1759-bug-ui-bug-fixes

1686-feat-implement-winston-for-logging

1536-bug-toggling-on-users-can-delete-workspaces-does-not-take-effect

agent-ui-mobile-styles

1522-feat-chromadb-support

1595-bug-unable-to-get-live-web-search-and-browsing-agent-working-using-google-custom-search-engine-error-getaddrinfo-enotfound-http-errno-3008

1582-bug-lm-studio-does-not-allow-for-different-model-selection

1312-bug-usernames-should-not-be-case-sensitive-when-logging-in

1029-feat-hf-serverless-inference-api

1086-feat-implement-normalized-input-fields

knowledge-graph-support

644-bug-uploaded-file-name-does-not-match-the-displayed-file-name-after-the-upload

v1.15.0

v1.14.2

v1.14.1

v1.14.0

v1.13.0

v1.12.1

v1.12.0

v1.11.2

v1.11.1

v1.11.0

v1.10.0

v1.9.1

v1.9.0

v1.8.5

v1.8.4

v1.8.3

v1.8.2

v1.8.1

v1.8.0

v1.7.8

v1.7.6

v1.7.5

v1.7.4

v1.4.0

v1.3.0

v1.2.4

v1.2.3

v1.2.2

v1.2.1

v1.2.0

v1.1.1

v1.1.0

v1.0.0

Labels

Clear labels

Desktop

Docker

Integration Request

Integration Request

OS: Linux

OS: Mobile

OS: Windows

UI/UX

blocked

bug

bug

core-team-only

documentation

duplicate

embed-widget

enhancement

feature request

github_actions

good first issue

investigating

needs info / can't replicate

possible bug

pull-request

Mirrored from GitHub Pull Request

question

stage: specifications

wontfix

No Label documentation

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

yindo

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#934