langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Alembic Offline Mode Fails Due to Data Migration Functions #16561

New Issue
Closed
opened 2026-02-21 19:26:40 -05:00 by yindo · 1 comment
yindo commented 2026-02-21 19:26:40 -05:00
Owner
Copy Link
Copy Source

Originally created by @QuantumGhost on GitHub (Aug 29, 2025).

Originally assigned to: @QuantumGhost on GitHub.

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Alembic offline mode (using the --sql flag) fails when processing migrations that contain data migration logic requiring active database connections. This prevents generating SQL scripts for deployment in environments where direct database access isn't available.

2. Additional context or comments

Steps to reproduce

Run the following bash command in api directory:

uv run flask db upgrade --sql

Related Migrations

Traceback

Traceback (most recent call last):
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/bin/flask", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask/cli.py", line 1131, in main
    cli.main()
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1363, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1226, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/decorators.py", line 34, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask/cli.py", line 400, in decorator
    return ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask_migrate/cli.py", line 154, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask_migrate/__init__.py", line 111, in wrapped
    f(*args, **kwargs)
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask_migrate/__init__.py", line 200, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/command.py", line 483, in upgrade
    script.run_env()
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/script/base.py", line 551, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 116, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 136, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/qg/workspace/langgenius/dify/api/migrations/env.py", line 107, in <module>
    run_migrations_offline()
  File "/Users/qg/workspace/langgenius/dify/api/migrations/env.py", line 70, in run_migrations_offline
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/runtime/environment.py", line 946, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
    step.migration_fn(**kw)
  File "/Users/qg/workspace/langgenius/dify/api/migrations/versions/2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py", line 46, in upgrade
    migrate_existing_providers_data()
  File "/Users/qg/workspace/langgenius/dify/api/migrations/versions/2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py", line 86, in migrate_existing_providers_data
    ).fetchall()
      ^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'fetchall'
2025/08/29 18:10:29 exit status 1

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @QuantumGhost on GitHub (Aug 29, 2025). Originally assigned to: @QuantumGhost on GitHub. ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Alembic offline mode (using the `--sql` flag) fails when processing migrations that contain data migration logic requiring active database connections. This prevents generating SQL scripts for deployment in environments where direct database access isn't available. ### 2. Additional context or comments ### Steps to reproduce Run the following bash command in `api` directory: ```bash uv run flask db upgrade --sql ``` ### Related Migrations - [2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py](https://github.com/langgenius/dify/blob/main/api/migrations/versions/2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py) - [2025_08_13_1605-0e154742a5fa_add_provider_model_multi_credential.py](https://github.com/langgenius/dify/blob/main/api/migrations/versions/2025_08_13_1605-0e154742a5fa_add_provider_model_multi_credential.py) ### Traceback ``` Traceback (most recent call last): File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/bin/flask", line 10, in <module> sys.exit(main()) ^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask/cli.py", line 1131, in main cli.main() File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1363, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1830, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1830, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 1226, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 794, in invoke return callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/decorators.py", line 34, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask/cli.py", line 400, in decorator return ctx.invoke(f, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/click/core.py", line 794, in invoke return callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask_migrate/cli.py", line 154, in upgrade _upgrade(directory, revision, sql, tag, x_arg) File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask_migrate/__init__.py", line 111, in wrapped f(*args, **kwargs) File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/flask_migrate/__init__.py", line 200, in upgrade command.upgrade(config, revision, sql=sql, tag=tag) File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/command.py", line 483, in upgrade script.run_env() File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/script/base.py", line 551, in run_env util.load_python_file(self.dir, "env.py") File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 116, in load_python_file module = load_module_py(module_id, path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 136, in load_module_py spec.loader.exec_module(module) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap_external>", line 999, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/Users/qg/workspace/langgenius/dify/api/migrations/env.py", line 107, in <module> run_migrations_offline() File "/Users/qg/workspace/langgenius/dify/api/migrations/env.py", line 70, in run_migrations_offline context.run_migrations() File "<string>", line 8, in run_migrations File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/runtime/environment.py", line 946, in run_migrations self.get_context().run_migrations(**kw) File "/Users/qg/workspace/langgenius/dify/temp/uv-project/.venv/lib/python3.12/site-packages/alembic/runtime/migration.py", line 627, in run_migrations step.migration_fn(**kw) File "/Users/qg/workspace/langgenius/dify/api/migrations/versions/2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py", line 46, in upgrade migrate_existing_providers_data() File "/Users/qg/workspace/langgenius/dify/api/migrations/versions/2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py", line 86, in migrate_existing_providers_data ).fetchall() ^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'fetchall' 2025/08/29 18:10:29 exit status 1 ``` ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.
yindo closed this issue 2026-02-21 19:26:40 -05:00
yindo commented 2026-02-21 19:26:40 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 29, 2025):

Alembic offline mode fails with your migration scripts because data migration logic (like querying and updating tables using op.get_bind()) requires an active database connection, which isn't available in offline mode—op.get_bind() returns None, so any attempt to execute queries (e.g., .fetchall()) throws an error. This is a known issue in Dify and Alembic generally, especially when schema and data migrations are mixed in the same script without guards for offline mode source.

Best practice: Always guard data migration code so it only runs in online mode. You can do this by checking context.is_offline_mode() (available via Alembic's migration context). For example:

from alembic import context

def upgrade():
    # schema changes here
    op.create_table(...)
    # Only run data migration if not in offline mode
    if not context.is_offline_mode():
        migrate_existing_providers_data()

This lets Alembic generate SQL scripts in offline mode without trying to run Python-based data migrations. For more complex cases, consider splitting schema and data migrations into separate migration files.

If you contribute a fix, update migration templates and docs to warn authors about this pattern and encourage using the guard. This will help future migrations work smoothly with both online and offline modes source.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Aug 29, 2025): <!-- Answer --> Alembic offline mode fails with your migration scripts because data migration logic (like querying and updating tables using `op.get_bind()`) requires an active database connection, which isn't available in offline mode—`op.get_bind()` returns `None`, so any attempt to execute queries (e.g., `.fetchall()`) throws an error. This is a known issue in Dify and Alembic generally, especially when schema and data migrations are mixed in the same script without guards for offline mode [source](https://github.com/langgenius/dify/blob/1a34ff8a67d5ccd6d9ffe46afde2a63dabbf9442/api/migrations/versions/2025_08_09_1553-e8446f481c1e_add_provider_credential_pool_support.py). **Best practice:** Always guard data migration code so it only runs in online mode. You can do this by checking `context.is_offline_mode()` (available via Alembic's migration context). For example: ```python from alembic import context def upgrade(): # schema changes here op.create_table(...) # Only run data migration if not in offline mode if not context.is_offline_mode(): migrate_existing_providers_data() ``` This lets Alembic generate SQL scripts in offline mode without trying to run Python-based data migrations. For more complex cases, consider splitting schema and data migrations into separate migration files. If you contribute a fix, update migration templates and docs to warn authors about this pattern and encourage using the guard. This will help future migrations work smoothly with both online and offline modes [source](https://github.com/langgenius/dify/blob/1a34ff8a67d5ccd6d9ffe46afde2a63dabbf9442/api/migrations/env.py). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/7bace54a-2a8c-4635-9867-23394d2ae08f?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/24794)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#16561
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?