langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-27 11:11:21 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Image files cannot be retrieved from Azure Blob storage #3999

New Issue
Closed
opened 2026-02-21 18:04:18 -05:00 by yindo · 8 comments
yindo commented 2026-02-21 18:04:18 -05:00
Owner
Copy Link
Copy Source

Originally created by @ysogabe on GitHub (Jun 6, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.10

Cloud or Self Hosted

Self Hosted (Docker), Self Hosted (Source)

Steps to reproduce

When using multimodal images and reloading the screen, the following error log appears and the images are not displayed

  • Storage： Azure Blob

logs

api-1 | [2024-06-06 04:32:03 +0000] [37] [ERROR] Error handling request /files/a9913e4b-a55e-4e25-b28b-6e76c5fd0130/image-preview?timestamp=1717648323&nonce=b81954d2ed2917cc29baeebcedffa00c&sign=6Ii7O0UWn5ebBJCHWcpzv06oYKYXlCCTuBl-eRjtgoM=
api-1 | Traceback (most recent call last):
api-1 | File "/app/api/extensions/storage/azure_storage.py", line 42, in generate
api-1 | while chunk := blob_stream.readall(4096):
api-1 | TypeError: StorageStreamDownloader.readall() takes 1 positional argument but 2 were given
api-1 |
api-1 | During handling of the above exception, another exception occurred:
api-1 |
api-1 | Traceback (most recent call last):
api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 55, in handle
api-1 | self.handle_request(listener_name, req, client, addr)
api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/ggevent.py", line 128, in handle_request
api-1 | super().handle_request(listener_name, req, sock, addr)
api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 115, in handle_request
api-1 | for item in respiter:
api-1 | File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 256, in next
api-1 | return self._next()
api-1 | File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 32, in _iter_encoded
api-1 | for item in iterable:
api-1 | File "/app/api/extensions/storage/azure_storage.py", line 41, in generate
api-1 | with closing(blob.download_blob()) as blob_stream:
api-1 | File "/usr/local/lib/python3.10/contextlib.py", line 340, in exit
api-1 | self.thing.close()
api-1 | AttributeError: 'StorageStreamDownloader' object has no attribute 'close'

✔️ Expected Behavior

No problems with image registration, image recognition, and processing of files to be registered in VectorDB using Azure Blob storage.

❌ Actual Behavior

Error only on redisplay of multimodal screen

Originally created by @ysogabe on GitHub (Jun 6, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.6.10 ### Cloud or Self Hosted Self Hosted (Docker), Self Hosted (Source) ### Steps to reproduce When using multimodal images and reloading the screen, the following error log appears and the images are not displayed - Storage： Azure Blob logs --- api-1 | [2024-06-06 04:32:03 +0000] [37] [ERROR] Error handling request /files/a9913e4b-a55e-4e25-b28b-6e76c5fd0130/image-preview?timestamp=1717648323&nonce=b81954d2ed2917cc29baeebcedffa00c&sign=6Ii7O0UWn5ebBJCHWcpzv06oYKYXlCCTuBl-eRjtgoM= api-1 | Traceback (most recent call last): api-1 | File "/app/api/extensions/storage/azure_storage.py", line 42, in generate api-1 | while chunk := blob_stream.readall(4096): api-1 | TypeError: StorageStreamDownloader.readall() takes 1 positional argument but 2 were given api-1 | api-1 | During handling of the above exception, another exception occurred: api-1 | api-1 | Traceback (most recent call last): api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 55, in handle api-1 | self.handle_request(listener_name, req, client, addr) api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/ggevent.py", line 128, in handle_request api-1 | super().handle_request(listener_name, req, sock, addr) api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 115, in handle_request api-1 | for item in respiter: api-1 | File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 256, in __next__ api-1 | return self._next() api-1 | File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 32, in _iter_encoded api-1 | for item in iterable: api-1 | File "/app/api/extensions/storage/azure_storage.py", line 41, in generate api-1 | with closing(blob.download_blob()) as blob_stream: api-1 | File "/usr/local/lib/python3.10/contextlib.py", line 340, in __exit__ api-1 | self.thing.close() api-1 | AttributeError: 'StorageStreamDownloader' object has no attribute 'close' ### ✔️ Expected Behavior No problems with image registration, image recognition, and processing of files to be registered in VectorDB using Azure Blob storage. ### ❌ Actual Behavior Error only on redisplay of multimodal screen
yindo added the 🐞 bug label 2026-02-21 18:04:18 -05:00
yindo closed this issue 2026-02-21 18:04:18 -05:00
yindo commented 2026-02-21 18:04:18 -05:00
Author
Owner
Copy Link
Copy Source

@ysogabe commented on GitHub (Jun 6, 2024):

After applying the following patch, it can be displayed without error

Patch：

--- a/api/extensions/storage/azure_storage.py
+++ b/api/extensions/storage/azure_storage.py
@@ -38,9 +38,8 @@ class AzureStorage(BaseStorage):
 
         def generate(filename: str = filename) -> Generator:
             blob = client.get_blob_client(container=self.bucket_name, blob=filename)
-            with closing(blob.download_blob()) as blob_stream:
-                while chunk := blob_stream.readall(4096):
-                    yield chunk
+            for chunk in blob.download_blob().chunks():
+                yield chunk
 
         return generate()
@ysogabe commented on GitHub (Jun 6, 2024): After applying the following patch, it can be displayed without error Patch： --- ```patch --- a/api/extensions/storage/azure_storage.py +++ b/api/extensions/storage/azure_storage.py @@ -38,9 +38,8 @@ class AzureStorage(BaseStorage): def generate(filename: str = filename) -> Generator: blob = client.get_blob_client(container=self.bucket_name, blob=filename) - with closing(blob.download_blob()) as blob_stream: - while chunk := blob_stream.readall(4096): - yield chunk + for chunk in blob.download_blob().chunks(): + yield chunk return generate() ```
yindo commented 2026-02-21 18:04:18 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 6, 2024):

Hello, @ysogabe! I'm here to help you with any bugs, questions, or contributions you may have. I'm Dosu, a bot designed to assist you while waiting for a human maintainer.

The issue arises because the StorageStreamDownloader.readall() method is being called with an incorrect number of arguments. The provided patch resolves this by using the chunks() method instead.

Here is the patch that fixes the issue:

--- a/api/extensions/storage/azure_storage.py
+++ b/api/extensions/storage/azure_storage.py
@@ -38,9 +38,8 @@ class AzureStorage(BaseStorage):
 
         def generate(filename: str = filename) -> Generator:
             blob = client.get_blob_client(container=self.bucket_name, blob=filename)
-            with closing(blob.download_blob()) as blob_stream:
-                while chunk := blob_stream.readall(4096):
-                    yield chunk
+            for chunk in blob.download_blob().chunks():
+                yield chunk
 
         return generate()

Applying this patch should resolve the error and allow images to be retrieved from Azure Blob Storage without issues [1].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Jun 6, 2024): <!-- Greeting --> Hello, @ysogabe! I'm here to help you with any bugs, questions, or contributions you may have. I'm Dosu, a bot designed to assist you while waiting for a human maintainer. <!-- Answer --> The issue arises because the `StorageStreamDownloader.readall()` method is being called with an incorrect number of arguments. The provided patch resolves this by using the `chunks()` method instead. Here is the patch that fixes the issue: ```patch --- a/api/extensions/storage/azure_storage.py +++ b/api/extensions/storage/azure_storage.py @@ -38,9 +38,8 @@ class AzureStorage(BaseStorage): def generate(filename: str = filename) -> Generator: blob = client.get_blob_client(container=self.bucket_name, blob=filename) - with closing(blob.download_blob()) as blob_stream: - while chunk := blob_stream.readall(4096): - yield chunk + for chunk in blob.download_blob().chunks(): + yield chunk return generate() ``` Applying this patch should resolve the error and allow images to be retrieved from Azure Blob Storage without issues <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/extensions/storage/azure_storage.py)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:04:18 -05:00
Author
Owner
Copy Link
Copy Source

@ysogabe commented on GitHub (Jun 6, 2024):

I tried #5004 version. My self host (source -> docker) , and it raises an exception if there is no close attribute.

api-1  | [2024-06-07 01:46:51 +0000] [37] [ERROR] Error handling request
api-1  | Traceback (most recent call last):
api-1  |   File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 725, in send
api-1  |     return self._sock.send(data, flags)
api-1  | BlockingIOError: [Errno 11] Resource temporarily unavailable
api-1  | 
api-1  | During handling of the above exception, another exception occurred:
api-1  | 
api-1  | Traceback (most recent call last):
api-1  |   File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 116, in handle_request
api-1  |     resp.write(item)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gunicorn/http/wsgi.py", line 355, in write
api-1  |     util.write(self.sock, arg, self.chunked)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gunicorn/util.py", line 298, in write
api-1  |     return write_chunk(sock, data)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gunicorn/util.py", line 293, in write_chunk
api-1  |     sock.sendall(chunk)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 702, in sendall
api-1  |     return _sendall(self, data_memory, flags)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 391, in _sendall
api-1  |     timeleft = __send_chunk(socket, chunk, flags, timeleft, end)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 320, in __send_chunk
api-1  |     data_sent += socket.send(chunk, flags)
api-1  |   File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 732, in send
api-1  |     return self._sock.send(data, flags)
api-1  | ConnectionResetError: [Errno 104] Connection reset by peer
api-1  | 
api-1  | During handling of the above exception, another exception occurred:
api-1  | 
api-1  | Traceback (most recent call last):
api-1  |   File "/app/api/extensions/storage/azure_storage.py", line 43, in generate
api-1  |     yield chunk
api-1  | GeneratorExit
api-1  | 
api-1  | During handling of the above exception, another exception occurred:
api-1  | 
api-1  | Traceback (most recent call last):
api-1  |   File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 122, in handle_request
api-1  |     respiter.close()
api-1  |   File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 260, in close
api-1  |     callback()
api-1  |   File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 403, in close
api-1  |     self.response.close()
api-1  |   File "/app/api/extensions/storage/azure_storage.py", line 41, in generate
api-1  |     with closing(blob.download_blob()) as blob_stream:
api-1  |   File "/usr/local/lib/python3.10/contextlib.py", line 340, in __exit__
api-1  |     self.thing.close()
api-1  | AttributeError: 'StorageStreamDownloader' object has no attribute 'close'
@ysogabe commented on GitHub (Jun 6, 2024): I tried #5004 version. My self host (source -> docker) , and it raises an exception if there is no close attribute. ```log api-1 | [2024-06-07 01:46:51 +0000] [37] [ERROR] Error handling request api-1 | Traceback (most recent call last): api-1 | File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 725, in send api-1 | return self._sock.send(data, flags) api-1 | BlockingIOError: [Errno 11] Resource temporarily unavailable api-1 | api-1 | During handling of the above exception, another exception occurred: api-1 | api-1 | Traceback (most recent call last): api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 116, in handle_request api-1 | resp.write(item) api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/http/wsgi.py", line 355, in write api-1 | util.write(self.sock, arg, self.chunked) api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/util.py", line 298, in write api-1 | return write_chunk(sock, data) api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/util.py", line 293, in write_chunk api-1 | sock.sendall(chunk) api-1 | File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 702, in sendall api-1 | return _sendall(self, data_memory, flags) api-1 | File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 391, in _sendall api-1 | timeleft = __send_chunk(socket, chunk, flags, timeleft, end) api-1 | File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 320, in __send_chunk api-1 | data_sent += socket.send(chunk, flags) api-1 | File "/usr/local/lib/python3.10/site-packages/gevent/_socketcommon.py", line 732, in send api-1 | return self._sock.send(data, flags) api-1 | ConnectionResetError: [Errno 104] Connection reset by peer api-1 | api-1 | During handling of the above exception, another exception occurred: api-1 | api-1 | Traceback (most recent call last): api-1 | File "/app/api/extensions/storage/azure_storage.py", line 43, in generate api-1 | yield chunk api-1 | GeneratorExit api-1 | api-1 | During handling of the above exception, another exception occurred: api-1 | api-1 | Traceback (most recent call last): api-1 | File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 122, in handle_request api-1 | respiter.close() api-1 | File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 260, in close api-1 | callback() api-1 | File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 403, in close api-1 | self.response.close() api-1 | File "/app/api/extensions/storage/azure_storage.py", line 41, in generate api-1 | with closing(blob.download_blob()) as blob_stream: api-1 | File "/usr/local/lib/python3.10/contextlib.py", line 340, in __exit__ api-1 | self.thing.close() api-1 | AttributeError: 'StorageStreamDownloader' object has no attribute 'close' ```
yindo commented 2026-02-21 18:04:18 -05:00
Author
Owner
Copy Link
Copy Source

@JohnJyong commented on GitHub (Jun 8, 2024):

it has fixed by below pr : https://github.com/langgenius/dify/pull/5004

@JohnJyong commented on GitHub (Jun 8, 2024): it has fixed by below pr : https://github.com/langgenius/dify/pull/5004
yindo commented 2026-02-21 18:04:19 -05:00
Author
Owner
Copy Link
Copy Source

@ysogabe commented on GitHub (Jun 25, 2024):

hi, @JohnJyong

Thanks for the response, I have tried pr #5004 and 0.6.11.
but, I am getting an error that the Close Attribute does not exist.

Changed azure_storage.py line 41 and it works correctly. Please consider changing it.

         def generate(filename: str = filename) -> Generator:
             blob = client.get_blob_client(container=self.bucket_name, blob=filename)
-            with closing(blob.download_blob()) as blob_stream:
-                while chunk := blob_stream.readall():
-                    yield chunk
+            for chunk in blob.download_blob().chunks():
+                yield chunk

  File "/app/api/extensions/storage/azure_storage.py", line 43, in generate
    yield chunk
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 122, in handle_request
    respiter.close()
  File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 260, in close
    callback()
  File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 403, in close
    self.response.close()
  File "/app/api/extensions/storage/azure_storage.py", line 41, in generate
    with closing(blob.download_blob()) as blob_stream:
  File "/usr/local/lib/python3.10/contextlib.py", line 340, in __exit__
    self.thing.close()
AttributeError: 'StorageStreamDownloader' object has no attribute 'close'
@ysogabe commented on GitHub (Jun 25, 2024): hi, @JohnJyong Thanks for the response, I have tried pr #5004 and 0.6.11. but, I am getting an error that the Close Attribute does not exist. Changed azure_storage.py line 41 and it works correctly. Please consider changing it. ```patch def generate(filename: str = filename) -> Generator: blob = client.get_blob_client(container=self.bucket_name, blob=filename) - with closing(blob.download_blob()) as blob_stream: - while chunk := blob_stream.readall(): - yield chunk + for chunk in blob.download_blob().chunks(): + yield chunk ``` ```log File "/app/api/extensions/storage/azure_storage.py", line 43, in generate yield chunk GeneratorExit During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/gunicorn/workers/base_async.py", line 122, in handle_request respiter.close() File "/usr/local/lib/python3.10/site-packages/werkzeug/wsgi.py", line 260, in close callback() File "/usr/local/lib/python3.10/site-packages/werkzeug/wrappers/response.py", line 403, in close self.response.close() File "/app/api/extensions/storage/azure_storage.py", line 41, in generate with closing(blob.download_blob()) as blob_stream: File "/usr/local/lib/python3.10/contextlib.py", line 340, in __exit__ self.thing.close() AttributeError: 'StorageStreamDownloader' object has no attribute 'close' ```
yindo commented 2026-02-21 18:04:19 -05:00
Author
Owner
Copy Link
Copy Source

@zolgear commented on GitHub (Jul 2, 2024):

Although it hasn't been determined whether Azure Blob is involved, enabling image uploads in the chat causes the browser to continuously download image streams until it displays an "Out of Memory" error when viewed on any screen.

The downloads stop either due to the "Out of Memory" error or a browser reload, but since tens of gigabytes are being downloaded each time, it significantly impacts the bandwidth charges.

If it turns out to be related, I will create an issue once confirmed.

@zolgear commented on GitHub (Jul 2, 2024): Although it hasn't been determined whether Azure Blob is involved, enabling image uploads in the chat causes the browser to continuously download image streams until it displays an "Out of Memory" error when viewed on any screen. The downloads stop either due to the "Out of Memory" error or a browser reload, but since tens of gigabytes are being downloaded each time, it significantly impacts the bandwidth charges. If it turns out to be related, I will create an issue once confirmed.
yindo commented 2026-02-21 18:04:19 -05:00
Author
Owner
Copy Link
Copy Source

@zolgear commented on GitHub (Jul 12, 2024):

The infinite download issue was fixed in version 0.6.13 by the changes made in #6063.

It seems that the issue with images not displaying in version 0.6.13 was fixed by #6196.

@zolgear commented on GitHub (Jul 12, 2024): The infinite download issue was fixed in version 0.6.13 by the changes made in #6063. It seems that the issue with images not displaying in version 0.6.13 was fixed by #6196.
yindo commented 2026-02-21 18:04:19 -05:00
Author
Owner
Copy Link
Copy Source

@ysogabe commented on GitHub (Jul 16, 2024):

@zolgear

I have tested the 0.6.13(14?) version and confirmed that the image files are retrieved and displayed correctly from Azure Blob Storage. Thank you for your support.

@ysogabe commented on GitHub (Jul 16, 2024): @zolgear I have tested the 0.6.13(14?) version and confirmed that the image files are retrieved and displayed correctly from Azure Blob Storage. Thank you for your support.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#3999
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?