langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #6702] Add docker-compose certbot configurations with backward compatibility #25395

New Issue
Closed
opened 2026-02-21 20:24:51 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:24:51 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/6702

State: closed
Merged: Yes

Checklist:

Important

Please review the checklist below before submitting your pull request.

  • Please open an issue before creating a PR or link to an existing issue
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Description

This PR adds docker-compose certbot configurations with backward compatibility. It introduces a new way to easily set up SSL certificates using Certbot while maintaining the existing method of using the nginx/ssl directory for those who prefer it. This change significantly simplifies the SSL setup process for new servers while ensuring that existing setups continue to work without modification.

The main benefits of this change are:

  1. Simplified SSL certificate acquisition and renewal process
  2. Improved security through automated certificate management
  3. Maintained backward compatibility for existing setups

This PR is a follow-up to #6509, addressing the request to implement SSL certificate management using Docker. While #6509 provided a guide for SSL setup using the host OS, this PR introduces a Docker-based solution using Certbot, making the process more streamlined and consistent with the project's containerized architecture.

Related to #6509

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update, included: Dify Document
  • Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
  • Dependency upgrade

Testing Instructions

Please follow the detailed test scenarios provided in the PR description. The main test scenarios are:

  • Test A: New feature with certbot container

    • Launch server properly
    • Test certbot certonly command
    • Verify certificate files work correctly
    • Test certbot renew command
    • Verify CERTBOT_OPTIONS functionality

  • Test B: Backward compatibility without certbot container

    • Verify legacy procedure still works with nginx/ssl directory

Detailed steps for each test scenario are provided in the PR description.

Add docker-compose certbot configurations description

Short description

Outline

  • Certbot available with --profile certbot option.
  • Without --profile certbot option you can still use cert files dir nginx/ssl as before.

Now that for newly launching servers SSL setup process is rather easy but still legacy way is effective.

See docker/certbot/README.md for easy setup.

Files editted/added

Document

file detail
README.md Updated, added to the section "What's Updated"
certbot/README.md Added desciption document

docker-compose:

file detail
docker-compse.yaml Updated nginx container settings, added certbot container settings
.env.example Added key-value pairs for certbot container (and consequently for nginx app)

nginx:

file detail
nginx/conf.d/default.conf.template Added acme-challenge location directive placeholder
nginx/https.conf.template Updated, simplified
nginx/docker-entrypoint.sh Updated, calculate ssl_certificate_path and ssl_certificate_key_path inside

certbot:

file detail
certbot/docker-entrypoint.sh added, generates certbot/update-cert.sh
certbot/update-cert.template added, template for certbot/update-cert.sh

update-cert.sh works as certbot certonly for the first time and later as certbot renew.
Personally I think some of certbot command options can be moved to CERTBOT_OPTIONS value but as I'm not sure which to moved would best so now CERTBOT_OPTIONS value is empty.

.env keys

Added keys below:

key default details
NGINX_ENABLE_CERTBOT_CHALLENGE false Set true to accept requests for /.well-known/acme-challenge/
CERTBOT_DOMAIN Domain name when use certbot container
CERTBOT_EMAIL Email address to use on certbot certonly certification
CERTBOT_OPTIONS Additional options for certbot command. i.e., --dry-run
Detailed Test Scenarios (click to expand)

Test scenarios for this update

Overview

This test scenarios assumes:

  • Dify app is installed in dir ~/dify/docker
  • No containers ever built or launched

Scenario1: New feature: with certbot container

  1. Test that the server launches properly
  2. Test that certbot certonly command works by running certbot/update-cert.sh
  3. Test that certificate files obtained by the certbot certonly command work correctly
  4. Test that certbot renew command works by running certbot/update-cert.sh
  5. Test that CERTBOT_OPTIONS values are correctly applied to the certbot command

Scenario2: Backward compatibility: without certbot container

  1. Test main branch procedure works
  2. Test that legacy procedure works

Details

Scenario1: New feature: with certbot container

Scenario1-1: Test that the server launches properly

Purpose:

  • Check that the server accepts normal http request.

Process overview:

  1. sudo docker-compose up
  2. Check that the server accepts normal http request
  3. sudo docker-compose down
1-1-1. sudo docker-compose up

Navigate to the dir ~/dify/docker and launch containers using docker-compose.yaml.

cd ~/dify/docker
sudo docker-compose up
1-1-2. Check that the server accepts normal http request

Then check server accessibility (HTTP)

http://your_domain.com

1-1-3. sudo docker-compose down

Then, docker-comose down

sudo docker-compose down

Scenario1-2: Test that certbot certonly command works by running certbot/update-cert.sh

Purpose:

  • Check that the server accepts requests for /.well-known/acme-challenge/
  • Check that by running certbot/update-cert.sh certbot certonly command works and successfully get cert files.

Process overview:

  1. Set .env values
  2. sudo docker-compose --profile certbot up
  3. Check cert dir status
  4. sudo docker-compose exec -it certbot /bin/sh /update-cert.sh
  5. Check the results if necessary
  6. sudo docker-compose down
1-2-1. Set .env values

Create a file ~/dify/docker/.env.

cd ~/dify/docker
vim .env

Add the line below and save the file.
(Or if you already copied .env.example to .env, edit the key below)

NGINX_ENABLE_CERTBOT_CHALLENGE=true
CERTBOT_DOMAIN=your_domain.com
CERTBOT_EMAIL=example@your_domain.com
1-2-2. sudo docker-compose --profile certbot up

Launch containers using docker-compose.yaml with option --profile certbot.

sudo docker network prune
sudo docker-compose --profile certbot up --force-recreate

First check that the server is accesabile using http protocol.

http://your_domain.com

1-2-3. Check cert dir status

Then, via another terminal:

Navigate to ~/dify/docker and check that no cert action excecuted yet.

cd ~/dify/docker
sudo docker-compose exec -it certbot ls /etc/letsencrypt/live/
sudo docker-compose exec -it certbot ls /var/log/letsencrypt/

No cert files and logs should be found.

1-2-4. sudo docker-compose exec -it certbot /bin/sh /update-cert.sh

Excecute command certbot certonly by executing /update-cert.sh

sudo docker-compose exec -it certbot /bin/sh /update-cert.sh

Expected succssful result as follows:

Certificate does not exist. Obtaining a new certificate...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Account registered.
Requesting a certificate for your_domain.com

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/your_domain.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/your_domain.com/privkey.pem
This certificate expires on 2024-10-23.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate operation successful
Please ensure to reload Nginx to apply any certificate changes.
1-2-5. Check the results if necessary

Check certbot logs if necesary:

cd ~/dify/docker
sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/
sudo docker-compose exec -it certbot ls -al /var/log/letsencrypt/
cat ~/dify/docker/volumes/certbot/logs/letsencrypt.log

1-2-6. sudo docker-compose down

Then docker-compose down

sudo docker-compose down

Scenario1-3: Test that certificate files obtained by the certbot certonly command work correctly

Purpose:

  • Check that the server accepts both http and https requests
  • Check that by running certbot/update-cert.sh certbot certonly command works and successfully get cert files.

Process overview:

  1. Set .env values
  2. sudo docker-compose --profile certbot up
  3. Check both http and https requests
  4. sudo docker-compose down
1-3-1. Set .env values

Edit .env file

cd ~/dify/docker
vim .env

Add the line below and save the file.
(Or if you already copied .env.example to .env, edit the key below)

# Add (or edit if already exists):
NGINX_HTTPS_ENABLED=true
NGINX_SSL_CERT_FILENAME=fullchain.pem
NGINX_SSL_CERT_KEY_FILENAME=privkey.pem
# Keys below already there:
NGINX_ENABLE_CERTBOT_CHALLENGE=true
CERTBOT_DOMAIN=your_domain.com
CERTBOT_EMAIL=example@your_domain.com
1-3-2. sudo docker-compose --profile certbot up

Launch containers using docker-compose.yaml with option --profile certbot.

sudo docker network prune
sudo docker-compose --profile certbot up --force-recreate
1-3-3. Check both http and https requests

Then check server accessibility (both http and https)

http://your_domain.com
https://your_domain.com

1-3-4. sudo docker-compose down

Then docker-compose down

sudo docker-compose down

Scenario1-4: Test that certbot renew command works by running certbot/update-cert.sh

Purpose:

  • Check that by running certbot/update-cert.sh certbot certonly command works and successfully get cert files.

Memo:
If the certificate already exists, certbot/update-cert.sh executes certbot renew.
This procedure is expected NOT to update cert files, as the cert files are not due for renewal.

Process overview:

  1. sudo docker-compose --profile certbot up
  2. Check cert files' timestamp
  3. sudo docker-compose exec -it certbot /bin/sh /update-cert.sh
  4. Check that timestamp for cert files DOES NOT changed
  5. Check both http and https requests
  6. sudo docker-compose down
1-4-1. sudo docker-compose --profile certbot up

Launch containers using docker-compose.yaml with option --profile certbot.

sudo docker-compose --profile certbot up
1-4-2. Check cert files' timestamp

Navigate to ~/dify/docker and check current cert files' timestamp:

cd ~/dify/docker
sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/

total 12
drwxr-xr-x    2 root     root          4096 Jul 25 22:06 .
drwxr-xr-x    3 root     root          4096 Jul 25 22:06 ..
-rw-r--r--    1 root     root           692 Jul 25 22:06 README
lrwxrwxrwx    1 root     root            38 Jul 25 22:06 cert.pem -> ../../archive/your_domain.com/cert1.pem
lrwxrwxrwx    1 root     root            39 Jul 25 22:06 chain.pem -> ../../archive/your_domain.com/chain1.pem
lrwxrwxrwx    1 root     root            43 Jul 25 22:06 fullchain.pem -> ../../archive/your_domain.com/fullchain1.pem
lrwxrwxrwx    1 root     root            41 Jul 25 22:06 privkey.pem -> ../../archive/your_domain.com/privkey1.pem
1-4-3. sudo docker-compose exec -it certbot /bin/sh /update-cert.sh

Excecute command certbot renew by executing /update-cert.sh

sudo docker-compose exec -it certbot /bin/sh /update-cert.sh

Expected succssful result as follows (No renewals were attempted as certs a not due for renewal).

Certificate exists. Attempting to renew...
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/your_domain.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate not yet due for renewal

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certificates are not due for renewal yet:
  /etc/letsencrypt/live/your_domain.com/fullchain.pem expires on 2024-10-23 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate operation successful
Please ensure to reload Nginx to apply any certificate changes.
1-4-4. Check that timestamp for cert files DOES NOT changed

Check that cert files not updated.

cd ~/dify/docker
sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/

total 12
drwxr-xr-x    2 root     root          4096 Jul 25 22:06 .
drwxr-xr-x    3 root     root          4096 Jul 25 22:06 ..
-rw-r--r--    1 root     root           692 Jul 25 22:06 README
lrwxrwxrwx    1 root     root            38 Jul 25 22:06 cert.pem -> ../../archive/your_domain.com/cert1.pem
lrwxrwxrwx    1 root     root            39 Jul 25 22:06 chain.pem -> ../../archive/your_domain.com/chain1.pem
lrwxrwxrwx    1 root     root            43 Jul 25 22:06 fullchain.pem -> ../../archive/your_domain.com/fullchain1.pem
lrwxrwxrwx    1 root     root            41 Jul 25 22:06 privkey.pem -> ../../archive/your_domain.com/privkey1.pem

Check certbot logs if necesary:

cat ~/dify/docker/volumes/certbot/logs/letsencrypt.log
1-4-5. Check both http and https requests

Then check server accessibility (both http and https)

http://your_domain.com
https://your_domain.com (also confirm that cert files are not updated)

1-4-6. sudo docker-compose down

Then docker-compose down

sudo docker-compose down

Scenario1-5: Test that CERTBOT_OPTIONS values are correctly applied to the certbot command

Purpose:

  • Check that CERTBOT_OPTIONS values are correctly applied to the certbot command

Process overview:

  1. Set .env values
  2. sudo docker-compose --profile certbot up
  3. Check cert files' timestamp
  4. sudo docker-compose exec -it certbot /bin/sh /update-cert.sh
  5. Check that timestamp for cert files DOES changed
  6. Check both http and https requests
  7. sudo docker-compose down
1-5-1. Set .env values

Edit .env file

vim .env

Add the line below and save the file.
(Or if you already copied .env.example to .env, edit the key below)

# Add (or edit if already exists):
CERTBOT_OPTIONS=--force-renewal
# Keys below already there:
NGINX_HTTPS_ENABLED=true
NGINX_SSL_CERT_FILENAME=fullchain.pem
NGINX_SSL_CERT_KEY_FILENAME=privkey.pem
NGINX_ENABLE_CERTBOT_CHALLENGE=true
CERTBOT_DOMAIN=your_domain.com
CERTBOT_EMAIL=example@your_domain.com
1-5-2. sudo docker-compose --profile certbot up

Launch containers using docker-compose.yaml with option --profile certbot.

sudo docker-compose --profile certbot up --force-recreate
1-5-3. Check cert files' timestamp

Navigate to ~/dify/docker and check current cert files' timestamp:

cd ~/dify/docker
sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/

total 12
drwxr-xr-x    2 root     root          4096 Jul 25 22:06 .
drwxr-xr-x    3 root     root          4096 Jul 25 22:06 ..
-rw-r--r--    1 root     root           692 Jul 25 22:06 README
lrwxrwxrwx    1 root     root            38 Jul 25 22:06 cert.pem -> ../../archive/your_domain.com/cert1.pem
lrwxrwxrwx    1 root     root            39 Jul 25 22:06 chain.pem -> ../../archive/your_domain.com/chain1.pem
lrwxrwxrwx    1 root     root            43 Jul 25 22:06 fullchain.pem -> ../../archive/your_domain.com/fullchain1.pem
lrwxrwxrwx    1 root     root            41 Jul 25 22:06 privkey.pem -> ../../archive/your_domain.com/privkey1.pem
1-5-4. sudo docker-compose exec -it certbot /bin/sh /update-cert.sh

Excecute command certbot renew by executing /update-cert.sh

sudo docker-compose exec -it certbot /bin/sh /update-cert.sh

Expected succssful result as follows (Updated even certs a not due for renewal).

webapp@ccc:~/dify/docker$ sudo docker-compose exec -it certbot /bin/sh /update-cert.sh
Certificate exists. Attempting to renew...
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/your_domain.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for your_domain.com

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded:
  /etc/letsencrypt/live/your_domain.com/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate operation successful
Please ensure to reload Nginx to apply any certificate changes.
1-5-5. Check that timestamp for cert files DOES changed

Check that cert files updated.

cd ~/dify/docker
sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/

total 12
drwxr-xr-x    2 root     root          4096 Jul 25 23:01 .
drwxr-xr-x    3 root     root          4096 Jul 25 22:06 ..
-rw-r--r--    1 root     root           692 Jul 25 22:06 README
lrwxrwxrwx    1 root     root            38 Jul 25 23:01 cert.pem -> ../../archive/your_domain.com/cert2.pem
lrwxrwxrwx    1 root     root            39 Jul 25 23:01 chain.pem -> ../../archive/your_domain.com/chain2.pem
lrwxrwxrwx    1 root     root            43 Jul 25 23:01 fullchain.pem -> ../../archive/your_domain.com/fullchain2.pem
lrwxrwxrwx    1 root     root            41 Jul 25 23:01 privkey.pem -> ../../archive/your_domain.com/privkey2.pem

Check certbot logs if necesary:

cat ~/dify/docker/volumes/certbot/logs/letsencrypt.log
1-5-6. Check both http and https requests

Then check renewed cert files effective by:

Reload Nginx

sudo docker-compose exec nginx nginx -s reload

Or restart docker containers

sudo docker-compose down
sudo docker network prune
sudo docker-compose --profile certbot up --force-recreate

Then check server accessibility (both http and https)

http://your_domain.com
https://your_domain.com

1-5-7. sudo docker-compose down

Then, it is recommended to set CERTBOT_OPTIONS value blank (or delete the key)

vim .env

Edit the line below and save the file.
(Or if you already copied .env.example to .env, edit the key below)

# Edit
CERTBOT_OPTIONS=""
# Keys below already there:
NGINX_HTTPS_ENABLED=true
NGINX_SSL_CERT_FILENAME=fullchain.pem
NGINX_SSL_CERT_KEY_FILENAME=privkey.pem
NGINX_ENABLE_CERTBOT_CHALLENGE=true
CERTBOT_DOMAIN=your_domain.com
CERTBOT_EMAIL=example@your_domain.com

Then docker-compose down

sudo docker-compose down

Scenario2: Backward compatibility: without certbot container

Memo:
Create a new server. Don't use the server used for the scenario test 1

Scenario2-1. Test that legacy procedure works

Purpose:
Confirm that main procedure works.
To confirm that migration process works using scenario 2-2.

Process overview:

  1. git checkout main
  2. Get cert files using host os certbot
  3. Copy cert files to docker/nginx/ssl
  4. sudo docker-compose WITHOUT --profile certbot
  5. Check both http and https requests
  6. sudo docker-compose down
2-1-1. git checkout main
cd ~/dify/docker
git checkout main
2-1-2. Get cert files using host os certbot
# Update system packages
sudo apt update

# Install Certbot
sudo apt install certbot

# Obtain SSL certificate (standalone mode)
sudo certbot certonly --standalone -d your_domain.com
2-1-3. Copy cert files to nginx/ssl/ and set read permission.
sudo ls -al /etc/letsencrypt/live/your_domain.com/

sudo cp -L /etc/letsencrypt/live/your_domain.com/{cert,chain,fullchain,privkey}.pem ~/dify/docker/nginx/ssl/
sudo mv ~/dify/docker/nginx/ssl/fullchain.pem ~/dify/docker/nginx/ssl/dify.crt
sudo mv ~/dify/docker/nginx/ssl/privkey.pem ~/dify/docker/nginx/ssl/dify.key

sudo chmod +r ~/dify/docker/nginx/ssl/*

ls -al ~/dify/docker/nginx/ssl/

Then you'll find that dify.crt and dify.key exists in the dirdocker/nginx/ssl/.

total 24
drwxrwxr-x 2 webapp webapp 4096 Jul 25 23:34 .
drwxrwxr-x 4 webapp webapp 4096 Jul 24 16:48 ..
-rw-rw-r-- 1 webapp webapp    0 Jul 24 09:45 .gitkeep
-rw-r--r-- 1 root   root   1273 Jul 25 23:34 cert.pem
-rw-r--r-- 1 root   root   1566 Jul 25 23:34 chain.pem
-rw-r--r-- 1 root   root   2839 Jul 25 23:34 dify.crt
-rw-r--r-- 1 root   root    241 Jul 25 23:34 dify.key
2-1-4. sudo docker-compose WITHOUT --profile certbot

Create a file ~/dify/docker/.env.

cd ~/dify/docker
vim .env

Add the line below and save the file.

# Add (or edit if already exists):
NGINX_HTTPS_ENABLED=true

Navigate to the dir ~/dify/docker and launch containers using docker-compose.yaml.

cd ~/dify/docker
sudo docker-compose up
2-1-5. Check both http and https requests

Then check server accessibility (both http and https)

http://your_domain.com
https://your_domain.com

2-1-6. sudo docker-compose down

Then docker-compose down

sudo docker-compose down

Scenario2-2. Test that legacy procedure works

Purpose:
Confirm that legacy docker/nginx/ssl storage also works as cert files location.

Process overview:

  1. git checkout this_branch
  2. sudo docker-compose with --profile certbot
  3. Check both http and https requests
  4. sudo docker-compose down
2-2-1. git checkout main
cd ~/dify/docker
git checkout this_branch
2-2-2. sudo docker-compose with --profile certbot

Navigate to the dir ~/dify/docker and launch containers using docker-compose.yaml.

cd ~/dify/docker
sudo docker-compose --profile certbot up --force-recreate
2-2-3. Check both http and https requests

Then check server accessibility (both http and https)

http://your_domain.com
https://your_domain.com

Then docker-compose down

2-2-4. sudo docker-compose down

Then docker-compose down

sudo docker-compose down
**Original Pull Request:** https://github.com/langgenius/dify/pull/6702 **State:** closed **Merged:** Yes --- # Checklist: > [!IMPORTANT] > Please review the checklist below before submitting your pull request. - [x] Please open an issue before creating a PR or link to an existing issue - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods # Description This PR adds docker-compose certbot configurations with backward compatibility. It introduces a new way to easily set up SSL certificates using Certbot while maintaining the existing method of using the `nginx/ssl` directory for those who prefer it. This change significantly simplifies the SSL setup process for new servers while ensuring that existing setups continue to work without modification. The main benefits of this change are: 1. Simplified SSL certificate acquisition and renewal process 2. Improved security through automated certificate management 3. Maintained backward compatibility for existing setups This PR is a follow-up to #6509, addressing the request to implement SSL certificate management using Docker. While #6509 provided a guide for SSL setup using the host OS, this PR introduces a Docker-based solution using Certbot, making the process more streamlined and consistent with the project's containerized architecture. Related to #6509 ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement - [ ] Dependency upgrade # Testing Instructions Please follow the detailed test scenarios provided in the PR description. The main test scenarios are: - [x] Test A: New feature with certbot container - Launch server properly - Test `certbot certonly` command - Verify certificate files work correctly - Test `certbot renew` command - Verify `CERTBOT_OPTIONS` functionality - [x] Test B: Backward compatibility without certbot container - Verify legacy procedure still works with `nginx/ssl` directory Detailed steps for each test scenario are provided in the PR description. *** # Add docker-compose certbot configurations description ## Short description ### Outline - Certbot available with `--profile certbot` option. - Without `--profile certbot` option you can still use cert files dir `nginx/ssl` as before. Now that for newly launching servers SSL setup process is rather easy but still legacy way is effective. See `docker/certbot/README.md` for easy setup. ## Files editted/added ### Document | file | detail | |---------------------|------------------------------------------------| | `README.md` | Updated, added to the section "What's Updated" | | `certbot/README.md` | Added desciption document | ### docker-compose: | file | detail | |----------------------|------------------------------------------------------------------------------| | `docker-compse.yaml` | Updated nginx container settings, added certbot container settings | | `.env.example` | Added key-value pairs for certbot container (and consequently for nginx app) | ### nginx: | file | detail | |--------------------------------------|-----------------------------------------------------------------------------| | `nginx/conf.d/default.conf.template` | Added acme-challenge location directive placeholder | | `nginx/https.conf.template` | Updated, simplified | | `nginx/docker-entrypoint.sh` | Updated, calculate ssl_certificate_path and ssl_certificate_key_path inside | ### certbot: | file | detail | |--------------------------------|--------------------------------------------| | `certbot/docker-entrypoint.sh` | added, generates certbot/update-cert.sh | | `certbot/update-cert.template` | added, template for certbot/update-cert.sh | > `update-cert.sh` works as `certbot certonly` for the first time and later as `certbot renew`. > Personally I think some of `certbot` command options can be moved to `CERTBOT_OPTIONS` value but as I'm not sure which to moved would best so now `CERTBOT_OPTIONS` value is empty. ### .env keys Added keys below: | key | default | details | |----------------------------------|---------|------------------------------------------------------------------| | `NGINX_ENABLE_CERTBOT_CHALLENGE` | `false` | Set `true` to accept requests for `/.well-known/acme-challenge/` | | `CERTBOT_DOMAIN` | | Domain name when use certbot container | | `CERTBOT_EMAIL` | | Email address to use on `certbot certonly` certification | | `CERTBOT_OPTIONS` | | Additional options for certbot command. i.e., `--dry-run` | *** <details> <summary>Detailed Test Scenarios (click to expand)</summary> ## Test scenarios for this update ## Overview This test scenarios assumes: - Dify app is installed in dir `~/dify/docker` - No containers ever built or launched ### Scenario1: New feature: with certbot container 1. Test that the server launches properly 2. Test that `certbot certonly` command works by running `certbot/update-cert.sh` 3. Test that certificate files obtained by the `certbot certonly` command work correctly 4. Test that `certbot renew` command works by running `certbot/update-cert.sh` 5. Test that `CERTBOT_OPTIONS` values are correctly applied to the `certbot` command ### Scenario2: Backward compatibility: without certbot container 1. Test main branch procedure works 2. Test that legacy procedure works ## Details ### Scenario1: New feature: with certbot container #### Scenario1-1: Test that the server launches properly > ***Purpose:*** > - Check that the server accepts normal http request. > > ***Process overview:*** > 1. `sudo docker-compose up` > 2. Check that the server accepts normal http request > 3. `sudo docker-compose down` ##### 1-1-1. `sudo docker-compose up` Navigate to the dir `~/dify/docker` and launch containers using `docker-compose.yaml`. ```shell cd ~/dify/docker sudo docker-compose up ``` ##### 1-1-2. Check that the server accepts normal http request Then check server accessibility (HTTP) http://your_domain.com ##### 1-1-3. `sudo docker-compose down` Then, `docker-comose down` ```shell sudo docker-compose down ``` #### Scenario1-2: Test that `certbot certonly` command works by running `certbot/update-cert.sh` > ***Purpose:*** > - Check that the server accepts requests for `/.well-known/acme-challenge/` > - Check that by running `certbot/update-cert.sh` `certbot certonly` command works and successfully get cert files. > > ***Process overview:*** > 1. Set `.env` values > 2. `sudo docker-compose --profile certbot up` > 3. Check cert dir status > 4. `sudo docker-compose exec -it certbot /bin/sh /update-cert.sh` > 4. Check the results if necessary > 5. `sudo docker-compose down` ##### 1-2-1. Set `.env` values Create a file `~/dify/docker/.env`. ```shell cd ~/dify/docker vim .env ``` Add the line below and save the file. (Or if you already copied .env.example to .env, edit the key below) ```properties NGINX_ENABLE_CERTBOT_CHALLENGE=true CERTBOT_DOMAIN=your_domain.com CERTBOT_EMAIL=example@your_domain.com ``` ##### 1-2-2. `sudo docker-compose --profile certbot up` Launch containers using `docker-compose.yaml` with option `--profile certbot`. ```shell sudo docker network prune sudo docker-compose --profile certbot up --force-recreate ``` First check that the server is accesabile using http protocol. http://your_domain.com ##### 1-2-3. Check cert dir status Then, via another terminal: Navigate to `~/dify/docker` and check that no cert action excecuted yet. ```shell cd ~/dify/docker sudo docker-compose exec -it certbot ls /etc/letsencrypt/live/ sudo docker-compose exec -it certbot ls /var/log/letsencrypt/ ``` No cert files and logs should be found. ##### 1-2-4. `sudo docker-compose exec -it certbot /bin/sh /update-cert.sh` Excecute command `certbot certonly` by executing `/update-cert.sh` ```shell sudo docker-compose exec -it certbot /bin/sh /update-cert.sh ``` Expected succssful result as follows: ```text Certificate does not exist. Obtaining a new certificate... Saving debug log to /var/log/letsencrypt/letsencrypt.log Account registered. Requesting a certificate for your_domain.com Successfully received certificate. Certificate is saved at: /etc/letsencrypt/live/your_domain.com/fullchain.pem Key is saved at: /etc/letsencrypt/live/your_domain.com/privkey.pem This certificate expires on 2024-10-23. These files will be updated when the certificate renews. NEXT STEPS: - The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If you like Certbot, please consider supporting our work by: * Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate * Donating to EFF: https://eff.org/donate-le - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Certificate operation successful Please ensure to reload Nginx to apply any certificate changes. ``` ##### 1-2-5. Check the results if necessary Check certbot logs if necesary: ```shell cd ~/dify/docker sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/ sudo docker-compose exec -it certbot ls -al /var/log/letsencrypt/ cat ~/dify/docker/volumes/certbot/logs/letsencrypt.log ``` ### 1-2-6. `sudo docker-compose down` Then `docker-compose down` ```shell sudo docker-compose down ``` #### Scenario1-3: Test that certificate files obtained by the `certbot certonly` command work correctly > ***Purpose:*** > - Check that the server accepts both http and https requests > - Check that by running `certbot/update-cert.sh` `certbot certonly` command works and successfully get cert files. > > ***Process overview:*** > 1. Set `.env` values > 2. `sudo docker-compose --profile certbot up` > 3. Check both http and https requests > 4. `sudo docker-compose down` ##### 1-3-1. Set `.env` values Edit `.env` file ```shell cd ~/dify/docker vim .env ``` Add the line below and save the file. (Or if you already copied .env.example to .env, edit the key below) ```properties # Add (or edit if already exists): NGINX_HTTPS_ENABLED=true NGINX_SSL_CERT_FILENAME=fullchain.pem NGINX_SSL_CERT_KEY_FILENAME=privkey.pem # Keys below already there: NGINX_ENABLE_CERTBOT_CHALLENGE=true CERTBOT_DOMAIN=your_domain.com CERTBOT_EMAIL=example@your_domain.com ``` ##### 1-3-2. `sudo docker-compose --profile certbot up` Launch containers using `docker-compose.yaml` with option `--profile certbot`. ```shell sudo docker network prune sudo docker-compose --profile certbot up --force-recreate ``` ##### 1-3-3. Check both http and https requests Then check server accessibility (both http and https) http://your_domain.com https://your_domain.com ##### 1-3-4. `sudo docker-compose down` Then `docker-compose down` ```shell sudo docker-compose down ``` #### Scenario1-4: Test that `certbot renew` command works by running `certbot/update-cert.sh` > ***Purpose:*** > - Check that by running `certbot/update-cert.sh` `certbot certonly` command works and successfully get cert files. > > ***Memo:*** > If the certificate already exists, `certbot/update-cert.sh` executes `certbot renew`. > ***This procedure is expected NOT to update cert files, as the cert files are not due for renewal.*** > > ***Process overview:*** > 1. `sudo docker-compose --profile certbot up` > 2. Check cert files' timestamp > 3. `sudo docker-compose exec -it certbot /bin/sh /update-cert.sh` > 4. Check that timestamp for cert files ***DOES NOT*** changed > 5. Check both http and https requests > 6. `sudo docker-compose down` ##### 1-4-1. `sudo docker-compose --profile certbot up` Launch containers using `docker-compose.yaml` with option `--profile certbot`. ```shell sudo docker-compose --profile certbot up ``` ##### 1-4-2. Check cert files' timestamp Navigate to `~/dify/docker` and check current cert files' timestamp: ```shell cd ~/dify/docker sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/ ``` ```text total 12 drwxr-xr-x 2 root root 4096 Jul 25 22:06 . drwxr-xr-x 3 root root 4096 Jul 25 22:06 .. -rw-r--r-- 1 root root 692 Jul 25 22:06 README lrwxrwxrwx 1 root root 38 Jul 25 22:06 cert.pem -> ../../archive/your_domain.com/cert1.pem lrwxrwxrwx 1 root root 39 Jul 25 22:06 chain.pem -> ../../archive/your_domain.com/chain1.pem lrwxrwxrwx 1 root root 43 Jul 25 22:06 fullchain.pem -> ../../archive/your_domain.com/fullchain1.pem lrwxrwxrwx 1 root root 41 Jul 25 22:06 privkey.pem -> ../../archive/your_domain.com/privkey1.pem ``` ##### 1-4-3. `sudo docker-compose exec -it certbot /bin/sh /update-cert.sh` Excecute command `certbot renew` by executing `/update-cert.sh` ```shell sudo docker-compose exec -it certbot /bin/sh /update-cert.sh ``` Expected succssful result as follows (No renewals were attempted as certs a not due for renewal). ```text Certificate exists. Attempting to renew... Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/your_domain.com.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Certificate not yet due for renewal - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The following certificates are not due for renewal yet: /etc/letsencrypt/live/your_domain.com/fullchain.pem expires on 2024-10-23 (skipped) No renewals were attempted. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Certificate operation successful Please ensure to reload Nginx to apply any certificate changes. ``` ##### 1-4-4. Check that timestamp for cert files ***DOES NOT*** changed Check that cert files not updated. ```shell cd ~/dify/docker sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/ ``` ```text total 12 drwxr-xr-x 2 root root 4096 Jul 25 22:06 . drwxr-xr-x 3 root root 4096 Jul 25 22:06 .. -rw-r--r-- 1 root root 692 Jul 25 22:06 README lrwxrwxrwx 1 root root 38 Jul 25 22:06 cert.pem -> ../../archive/your_domain.com/cert1.pem lrwxrwxrwx 1 root root 39 Jul 25 22:06 chain.pem -> ../../archive/your_domain.com/chain1.pem lrwxrwxrwx 1 root root 43 Jul 25 22:06 fullchain.pem -> ../../archive/your_domain.com/fullchain1.pem lrwxrwxrwx 1 root root 41 Jul 25 22:06 privkey.pem -> ../../archive/your_domain.com/privkey1.pem ``` > Check certbot logs if necesary: > ```shell > cat ~/dify/docker/volumes/certbot/logs/letsencrypt.log > ``` ##### 1-4-5. Check both http and https requests Then check server accessibility (both http and https) http://your_domain.com https://your_domain.com (also confirm that cert files are not updated) ##### 1-4-6. `sudo docker-compose down` Then `docker-compose down` ```shell sudo docker-compose down ``` #### Scenario1-5: Test that `CERTBOT_OPTIONS` values are correctly applied to the `certbot` command > ***Purpose:*** > - Check that `CERTBOT_OPTIONS` values are correctly applied to the `certbot` command > > ***Process overview:*** > 1. Set `.env` values > 2. `sudo docker-compose --profile certbot up` > 3. Check cert files' timestamp > 4. `sudo docker-compose exec -it certbot /bin/sh /update-cert.sh` > 5. Check that timestamp for cert files ***DOES*** changed > 6. Check both http and https requests > 7. `sudo docker-compose down` ##### 1-5-1. Set `.env` values Edit `.env` file ```shell vim .env ``` Add the line below and save the file. (Or if you already copied .env.example to .env, edit the key below) ```properties # Add (or edit if already exists): CERTBOT_OPTIONS=--force-renewal # Keys below already there: NGINX_HTTPS_ENABLED=true NGINX_SSL_CERT_FILENAME=fullchain.pem NGINX_SSL_CERT_KEY_FILENAME=privkey.pem NGINX_ENABLE_CERTBOT_CHALLENGE=true CERTBOT_DOMAIN=your_domain.com CERTBOT_EMAIL=example@your_domain.com ``` ##### 1-5-2. `sudo docker-compose --profile certbot up` Launch containers using `docker-compose.yaml` with option `--profile certbot`. ```shell sudo docker-compose --profile certbot up --force-recreate ``` ##### 1-5-3. Check cert files' timestamp Navigate to `~/dify/docker` and check current cert files' timestamp: ```shell cd ~/dify/docker sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/ ``` ```text total 12 drwxr-xr-x 2 root root 4096 Jul 25 22:06 . drwxr-xr-x 3 root root 4096 Jul 25 22:06 .. -rw-r--r-- 1 root root 692 Jul 25 22:06 README lrwxrwxrwx 1 root root 38 Jul 25 22:06 cert.pem -> ../../archive/your_domain.com/cert1.pem lrwxrwxrwx 1 root root 39 Jul 25 22:06 chain.pem -> ../../archive/your_domain.com/chain1.pem lrwxrwxrwx 1 root root 43 Jul 25 22:06 fullchain.pem -> ../../archive/your_domain.com/fullchain1.pem lrwxrwxrwx 1 root root 41 Jul 25 22:06 privkey.pem -> ../../archive/your_domain.com/privkey1.pem ``` ##### 1-5-4. `sudo docker-compose exec -it certbot /bin/sh /update-cert.sh` Excecute command `certbot renew` by executing `/update-cert.sh` ```shell sudo docker-compose exec -it certbot /bin/sh /update-cert.sh ``` Expected succssful result as follows (Updated even certs a not due for renewal). ```text webapp@ccc:~/dify/docker$ sudo docker-compose exec -it certbot /bin/sh /update-cert.sh Certificate exists. Attempting to renew... Saving debug log to /var/log/letsencrypt/letsencrypt.log - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/your_domain.com.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Renewing an existing certificate for your_domain.com - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Congratulations, all renewals succeeded: /etc/letsencrypt/live/your_domain.com/fullchain.pem (success) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Certificate operation successful Please ensure to reload Nginx to apply any certificate changes. ``` ##### 1-5-5. Check that timestamp for cert files ***DOES*** changed Check that cert files updated. ```shell cd ~/dify/docker sudo docker-compose exec -it certbot ls -al /etc/letsencrypt/live/your_domain.com/ ``` ```text total 12 drwxr-xr-x 2 root root 4096 Jul 25 23:01 . drwxr-xr-x 3 root root 4096 Jul 25 22:06 .. -rw-r--r-- 1 root root 692 Jul 25 22:06 README lrwxrwxrwx 1 root root 38 Jul 25 23:01 cert.pem -> ../../archive/your_domain.com/cert2.pem lrwxrwxrwx 1 root root 39 Jul 25 23:01 chain.pem -> ../../archive/your_domain.com/chain2.pem lrwxrwxrwx 1 root root 43 Jul 25 23:01 fullchain.pem -> ../../archive/your_domain.com/fullchain2.pem lrwxrwxrwx 1 root root 41 Jul 25 23:01 privkey.pem -> ../../archive/your_domain.com/privkey2.pem ``` > Check certbot logs if necesary: > ```shell > cat ~/dify/docker/volumes/certbot/logs/letsencrypt.log > ``` ##### 1-5-6. Check both http and https requests Then check renewed cert files effective by: Reload Nginx ```shell sudo docker-compose exec nginx nginx -s reload ``` Or restart docker containers ```shell sudo docker-compose down sudo docker network prune sudo docker-compose --profile certbot up --force-recreate ``` Then check server accessibility (both http and https) http://your_domain.com https://your_domain.com ##### 1-5-7. `sudo docker-compose down` Then, it is recommended to set `CERTBOT_OPTIONS` value blank (or delete the key) ```shell vim .env ``` Edit the line below and save the file. (Or if you already copied .env.example to .env, edit the key below) ```properties # Edit CERTBOT_OPTIONS="" # Keys below already there: NGINX_HTTPS_ENABLED=true NGINX_SSL_CERT_FILENAME=fullchain.pem NGINX_SSL_CERT_KEY_FILENAME=privkey.pem NGINX_ENABLE_CERTBOT_CHALLENGE=true CERTBOT_DOMAIN=your_domain.com CERTBOT_EMAIL=example@your_domain.com ``` Then `docker-compose down` ```shell sudo docker-compose down ``` ### Scenario2: Backward compatibility: without certbot container > ***Memo:*** > Create a new server. ***Don't use the server used for the scenario test 1*** #### Scenario2-1. Test that legacy procedure works > ***Purpose:*** > Confirm that main procedure works. > To confirm that migration process works using scenario 2-2. > ***Process overview:*** > 1. git checkout main > 2. Get cert files using host os certbot > 3. Copy cert files to `docker/nginx/ssl` > 4. `sudo docker-compose` WITHOUT `--profile certbot` > 5. Check both http and https requests > 5. `sudo docker-compose down` ##### 2-1-1. git checkout main ```shell cd ~/dify/docker git checkout main ``` ##### 2-1-2. Get cert files using host os certbot ```bash # Update system packages sudo apt update # Install Certbot sudo apt install certbot # Obtain SSL certificate (standalone mode) sudo certbot certonly --standalone -d your_domain.com ``` ##### 2-1-3. Copy cert files to `nginx/ssl/` and set read permission. ``` sudo ls -al /etc/letsencrypt/live/your_domain.com/ sudo cp -L /etc/letsencrypt/live/your_domain.com/{cert,chain,fullchain,privkey}.pem ~/dify/docker/nginx/ssl/ sudo mv ~/dify/docker/nginx/ssl/fullchain.pem ~/dify/docker/nginx/ssl/dify.crt sudo mv ~/dify/docker/nginx/ssl/privkey.pem ~/dify/docker/nginx/ssl/dify.key sudo chmod +r ~/dify/docker/nginx/ssl/* ls -al ~/dify/docker/nginx/ssl/ ``` Then you'll find that `dify.crt` and `dify.key` exists in the dir`docker/nginx/ssl/`. ```text total 24 drwxrwxr-x 2 webapp webapp 4096 Jul 25 23:34 . drwxrwxr-x 4 webapp webapp 4096 Jul 24 16:48 .. -rw-rw-r-- 1 webapp webapp 0 Jul 24 09:45 .gitkeep -rw-r--r-- 1 root root 1273 Jul 25 23:34 cert.pem -rw-r--r-- 1 root root 1566 Jul 25 23:34 chain.pem -rw-r--r-- 1 root root 2839 Jul 25 23:34 dify.crt -rw-r--r-- 1 root root 241 Jul 25 23:34 dify.key ``` ##### 2-1-4. `sudo docker-compose` WITHOUT `--profile certbot` Create a file `~/dify/docker/.env`. ```shell cd ~/dify/docker vim .env ``` Add the line below and save the file. ```properties # Add (or edit if already exists): NGINX_HTTPS_ENABLED=true ``` Navigate to the dir `~/dify/docker` and launch containers using `docker-compose.yaml`. ```shell cd ~/dify/docker sudo docker-compose up ``` ##### 2-1-5. Check both http and https requests Then check server accessibility (both http and https) http://your_domain.com https://your_domain.com ##### 2-1-6. `sudo docker-compose down` Then `docker-compose down` ```shell sudo docker-compose down ``` #### Scenario2-2. Test that legacy procedure works > ***Purpose:*** > Confirm that legacy `docker/nginx/ssl` storage also works as cert files location. > > ***Process overview:*** > 1. git checkout this_branch > 2. `sudo docker-compose` with `--profile certbot` > 3. Check both http and https requests > 4. `sudo docker-compose down` ##### 2-2-1. git checkout main ```shell cd ~/dify/docker git checkout this_branch ``` ##### 2-2-2. `sudo docker-compose` with `--profile certbot` Navigate to the dir `~/dify/docker` and launch containers using `docker-compose.yaml`. ```shell cd ~/dify/docker sudo docker-compose --profile certbot up --force-recreate ``` ##### 2-2-3. Check both http and https requests Then check server accessibility (both http and https) http://your_domain.com https://your_domain.com Then `docker-compose down` ##### 2-2-4. `sudo docker-compose down` Then `docker-compose down` ```shell sudo docker-compose down ``` </details>
yindo added the pull-request label 2026-02-21 20:24:51 -05:00
yindo closed this issue 2026-02-21 20:24:51 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#25395
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?