langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-24 22:11:36 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

HA and Autoscale for AWS deployment #5412

New Issue
Closed
opened 2026-02-21 18:10:51 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 18:10:51 -05:00
Owner
Copy Link
Copy Source

Originally created by @imdevopx on GitHub (Sep 5, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

slowness and timeout while running workflows

2. Additional context or comments

currently we have one container each service deployed in aws ECS, is it possible for us to run 2 containers each services? and autoscale if ever needed.

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @imdevopx on GitHub (Sep 5, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. slowness and timeout while running workflows ### 2. Additional context or comments currently we have one container each service deployed in aws ECS, is it possible for us to run 2 containers each services? and autoscale if ever needed. ### 3. Can you help us with this feature? - [ ] I am interested in contributing to this feature.
yindo added the 💪 enhancement label 2026-02-21 18:10:51 -05:00
yindo closed this issue 2026-02-21 18:10:51 -05:00
yindo commented 2026-02-21 18:10:51 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Sep 5, 2024):

Yes, it is possible to run multiple containers for each service and enable autoscaling in AWS ECS. Here are some key points to consider:

  1. Service Dependencies:

    • Ensure that all service dependencies are properly configured. For example, the api and worker services depend on db and redis.

  2. Scaling Configuration:

    • You can configure ECS Service Auto Scaling to automatically adjust the number of container instances in your ECS service. This can be done by setting up scaling policies based on CloudWatch metrics such as CPU utilization, memory usage, or custom metrics.

  3. Load Balancing:

    • Use an Application Load Balancer (ALB) to distribute traffic across multiple containers. This ensures high availability and better performance.

  4. Task Definitions:

    • Update your ECS task definitions to specify the desired number of tasks (containers) for each service. You can set the desiredCount parameter to the number of containers you want to run.

  5. Cluster Configuration:

    • Ensure your ECS cluster has enough resources (CPU, memory) to handle the additional containers. You may need to adjust the cluster size or instance types.

Here is an example of how you might configure the api service to run multiple containers and enable autoscaling:

api:
  image: langgenius/dify-api:0.7.3
  restart: always
  environment:
    <<: *shared-api-worker-env
    MODE: api
  depends_on:
    - db
    - redis
  volumes:
    - ./volumes/app/storage:/app/api/storage
  networks:
    - ssrf_proxy_network
    - default
  deploy:
    replicas: 2
    update_config:
      parallelism: 2
      delay: 10s
    resources:
      limits:
        cpus: '0.50'
        memory: 512M
      reservations:
        cpus: '0.25'
        memory: 256M
    restart_policy:
      condition: on-failure

For autoscaling, you can use the AWS Management Console or AWS CLI to create scaling policies. Here is an example using AWS CLI:

aws application-autoscaling register-scalable-target \
    --service-namespace ecs \
    --resource-id service/your-cluster-name/your-service-name \
    --scalable-dimension ecs:service:DesiredCount \
    --min-capacity 1 \
    --max-capacity 10

aws application-autoscaling put-scaling-policy \
    --service-namespace ecs \
    --scalable-dimension ecs:service:DesiredCount \
    --resource-id service/your-cluster-name/your-service-name \
    --policy-name cpu-utilization-policy \
    --policy-type TargetTrackingScaling \
    --target-tracking-scaling-policy-configuration file://scaling-policy.json

In the scaling-policy.json file, you can define the target tracking configuration:

{
  "TargetValue": 50.0,
  "PredefinedMetricSpecification": {
    "PredefinedMetricType": "ECSServiceAverageCPUUtilization"
  },
  "ScaleOutCooldown": 60,
  "ScaleInCooldown": 60
}

This setup will help you run multiple containers for each service and enable autoscaling based on CPU utilization. Make sure to adjust the configurations according to your specific requirements and resource limits [1][2].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Sep 5, 2024): <!-- Answer --> Yes, it is possible to run multiple containers for each service and enable autoscaling in AWS ECS. Here are some key points to consider: 1. **Service Dependencies**: - Ensure that all service dependencies are properly configured. For example, the `api` and `worker` services depend on `db` and `redis`. 2. **Scaling Configuration**: - You can configure ECS Service Auto Scaling to automatically adjust the number of container instances in your ECS service. This can be done by setting up scaling policies based on CloudWatch metrics such as CPU utilization, memory usage, or custom metrics. 3. **Load Balancing**: - Use an Application Load Balancer (ALB) to distribute traffic across multiple containers. This ensures high availability and better performance. 4. **Task Definitions**: - Update your ECS task definitions to specify the desired number of tasks (containers) for each service. You can set the `desiredCount` parameter to the number of containers you want to run. 5. **Cluster Configuration**: - Ensure your ECS cluster has enough resources (CPU, memory) to handle the additional containers. You may need to adjust the cluster size or instance types. Here is an example of how you might configure the `api` service to run multiple containers and enable autoscaling: ```yaml api: image: langgenius/dify-api:0.7.3 restart: always environment: <<: *shared-api-worker-env MODE: api depends_on: - db - redis volumes: - ./volumes/app/storage:/app/api/storage networks: - ssrf_proxy_network - default deploy: replicas: 2 update_config: parallelism: 2 delay: 10s resources: limits: cpus: '0.50' memory: 512M reservations: cpus: '0.25' memory: 256M restart_policy: condition: on-failure ``` For autoscaling, you can use the AWS Management Console or AWS CLI to create scaling policies. Here is an example using AWS CLI: ```sh aws application-autoscaling register-scalable-target \ --service-namespace ecs \ --resource-id service/your-cluster-name/your-service-name \ --scalable-dimension ecs:service:DesiredCount \ --min-capacity 1 \ --max-capacity 10 aws application-autoscaling put-scaling-policy \ --service-namespace ecs \ --scalable-dimension ecs:service:DesiredCount \ --resource-id service/your-cluster-name/your-service-name \ --policy-name cpu-utilization-policy \ --policy-type TargetTrackingScaling \ --target-tracking-scaling-policy-configuration file://scaling-policy.json ``` In the `scaling-policy.json` file, you can define the target tracking configuration: ```json { "TargetValue": 50.0, "PredefinedMetricSpecification": { "PredefinedMetricType": "ECSServiceAverageCPUUtilization" }, "ScaleOutCooldown": 60, "ScaleInCooldown": 60 } ``` This setup will help you run multiple containers for each service and enable autoscaling based on CPU utilization. Make sure to adjust the configurations according to your specific requirements and resource limits <sup>[[1]](https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml)</sup><sup>[[2]](https://github.com/langgenius/dify/blob/main/docker/README.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:10:51 -05:00
Author
Owner
Copy Link
Copy Source

@imdevopx commented on GitHub (Sep 11, 2024):

seems weaviate doesn't support replica? got error in weaviate after doing autoscale:
"error: create index: init shard: shard db: create objects bucket: recover commit log: context deadline exceeded."

@imdevopx commented on GitHub (Sep 11, 2024): seems weaviate doesn't support replica? got error in weaviate after doing autoscale: "error: create index: init shard: shard db: create objects bucket: recover commit log: context deadline exceeded."
yindo commented 2026-02-21 18:10:52 -05:00
Author
Owner
Copy Link
Copy Source

@xjiaqing commented on GitHub (Sep 17, 2024):

@imdevopx Hi, I'm try to deploy dify in Amazon ECS, but failed. Do you have any guidence about it?

@xjiaqing commented on GitHub (Sep 17, 2024): @imdevopx Hi, I'm try to deploy dify in Amazon ECS, but failed. Do you have any guidence about it?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5412
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?