langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-27 03:01:54 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

The mime type of the signed url in s3 is not being retrieved by dify in "remote_url". #6785

New Issue
Closed
opened 2026-02-21 18:17:17 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 18:17:17 -05:00
Owner
Copy Link
Copy Source

Originally created by @kitotakumi on GitHub (Nov 15, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I am passing a signed s3 url from lambda to dify with “remote_url”. When doing so, the LLM returns an error stating that the mime_type is blank. This is because the mime_type is not obtained by theify, although I do mimetype.guess_type and resp.headers.get(“Content-Type”, “”) in api/factories/file_factory.py, I am not getting the mime_type in either case.

In case of request, I can get mime_type by placing get instead of head. I would like it to be possible to obtain mime_type even if head cannot be obtained.

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @kitotakumi on GitHub (Nov 15, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. I am passing a signed s3 url from lambda to dify with “remote_url”. When doing so, the LLM returns an error stating that the mime_type is blank. This is because the mime_type is not obtained by theify, although I do mimetype.guess_type and resp.headers.get(“Content-Type”, “”) in api/factories/file_factory.py, I am not getting the mime_type in either case. In case of request, I can get mime_type by placing get instead of head. I would like it to be possible to obtain mime_type even if head cannot be obtained. ### 2. Additional context or comments _No response_ ### 3. Can you help us with this feature? - [X] I am interested in contributing to this feature.
yindo added the 🐞 bug label 2026-02-21 18:17:17 -05:00
yindo closed this issue 2026-02-21 18:17:17 -05:00
yindo commented 2026-02-21 18:17:18 -05:00
Author
Owner
Copy Link
Copy Source

@kitotakumi commented on GitHub (Nov 15, 2024):

Now that I know the cause, I will create my own pull request.

@kitotakumi commented on GitHub (Nov 15, 2024): Now that I know the cause, I will create my own pull request.
yindo commented 2026-02-21 18:17:18 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 15, 2024):

Hey @kitotakumi! I'm here to help you with any bugs, questions, or contributions you have. Let's work together to solve this! Dosu

To address the issue of not retrieving the mime type from a signed S3 URL in Dify's "remote_url," you can modify the approach to use a GET request as a fallback when the HEAD request does not return the necessary headers. This method is already implemented in the RemoteFileInfoApi class in api/controllers/console/remote_files.py. Here's how it works:

  1. Attempt to retrieve the mime type using a HEAD request.
  2. If the HEAD request fails to provide the Content-Type, fall back to a GET request to obtain the necessary information.

Here's a snippet of how this is implemented:

class RemoteFileInfoApi(Resource):
    @marshal_with(remote_file_info_fields)
    def get(self, url):
        decoded_url = urllib.parse.unquote(url)
        resp = ssrf_proxy.head(decoded_url)
        if resp.status_code != httpx.codes.OK:
            # Fallback to GET method
            resp = ssrf_proxy.get(decoded_url, timeout=3)
        resp.raise_for_status()
        return {
            "file_type": resp.headers.get("Content-Type", "application/octet-stream"),
            "file_length": int(resp.headers.get("Content-Length", 0)),
        }

This approach ensures that even if the HEAD request does not provide the Content-Type, the GET request will be used as a fallback to retrieve the mime type [1].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Nov 15, 2024): <!-- Greeting --> Hey @kitotakumi! I'm here to help you with any bugs, questions, or contributions you have. Let's work together to solve this! [Dosu](https://dosu.dev) <!-- Answer --> To address the issue of not retrieving the mime type from a signed S3 URL in Dify's "remote_url," you can modify the approach to use a `GET` request as a fallback when the `HEAD` request does not return the necessary headers. This method is already implemented in the `RemoteFileInfoApi` class in `api/controllers/console/remote_files.py`. Here's how it works: 1. Attempt to retrieve the mime type using a `HEAD` request. 2. If the `HEAD` request fails to provide the `Content-Type`, fall back to a `GET` request to obtain the necessary information. Here's a snippet of how this is implemented: ```python class RemoteFileInfoApi(Resource): @marshal_with(remote_file_info_fields) def get(self, url): decoded_url = urllib.parse.unquote(url) resp = ssrf_proxy.head(decoded_url) if resp.status_code != httpx.codes.OK: # Fallback to GET method resp = ssrf_proxy.get(decoded_url, timeout=3) resp.raise_for_status() return { "file_type": resp.headers.get("Content-Type", "application/octet-stream"), "file_length": int(resp.headers.get("Content-Length", 0)), } ``` This approach ensures that even if the `HEAD` request does not provide the `Content-Type`, the `GET` request will be used as a fallback to retrieve the mime type <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/controllers/console/remote_files.py)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:17:18 -05:00
Author
Owner
Copy Link
Copy Source

@kitotakumi commented on GitHub (Nov 15, 2024):

Thank you very much. It is indeed possible to get the mime_type correctly by using get. However, I think it is better to get the mime_type from filename, which is implemented in api/controllers/common/helpers.py. We have also confirmed that it works correctly. We would like to make a fix using one of those two methods.
mimetype, _ = mimetypes.guess_type(filename)

@kitotakumi commented on GitHub (Nov 15, 2024): Thank you very much. It is indeed possible to get the mime_type correctly by using get. However, I think it is better to get the mime_type from filename, which is implemented in api/controllers/common/helpers.py. We have also confirmed that it works correctly. We would like to make a fix using one of those two methods. `mimetype, _ = mimetypes.guess_type(filename)`
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6785
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?