langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-08-27 11:11:21 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Add PDF Processing Support for Vertex AI Gemini Provider with Graph Recognition #7387

New Issue
Closed
opened 2026-02-21 18:20:21 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 18:20:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @wmoto-ai on GitHub (Dec 25, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Currently, while Google AI Studio provider in Dify supports direct PDF processing, the Vertex AI Gemini provider lacks this capability. This creates a significant limitation for users who need to process PDFs containing graphs and figures but are required to use Vertex AI for policy/security reasons. The current workaround requires separate processing of text and images before feeding them to the LLM, which is inefficient.

What is the feature you'd like to see?
Implementation of direct PDF processing capability in the Vertex AI Gemini provider, similar to what's available in Google AI Studio provider. The feature should:

  • Enable direct upload and processing of PDF files
  • Support extraction and interpretation of both text and visual elements (especially graphs)
  • Handle PDF content as proper glm.Part with mime_type=application/pdf
  • Maintain the multimodal capabilities of Gemini for analyzing graphs and figures within PDFs

How will this feature improve your workflow / experience?
This feature will significantly streamline the workflow by:

  • Eliminating the need for separate processing of text and images from PDFs
  • Enabling direct analysis of graphs and figures within documents
  • Reducing the manual effort required to prepare PDF content for LLM processing
  • Making the Vertex AI provider as capable as Google AI Studio in terms of PDF handling

2. Additional context or comments

  • Special emphasis on graph recognition and interpretation capabilities
  • Technical implementation might involve:
    • Adding 'document' feature to gemini-*.yaml
    • Modifying llm.py to handle PDF documents as glm.Part
    • Ensuring proper MIME type handling for PDFs
    • Previous implementation attempts have been made but were unsuccessful in getting the PDF processing to work correctly

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
Originally created by @wmoto-ai on GitHub (Dec 25, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Currently, while Google AI Studio provider in Dify supports direct PDF processing, the Vertex AI Gemini provider lacks this capability. This creates a significant limitation for users who need to process PDFs containing graphs and figures but are required to use Vertex AI for policy/security reasons. The current workaround requires separate processing of text and images before feeding them to the LLM, which is inefficient. **What is the feature you'd like to see?** Implementation of direct PDF processing capability in the Vertex AI Gemini provider, similar to what's available in Google AI Studio provider. The feature should: - Enable direct upload and processing of PDF files - Support extraction and interpretation of both text and visual elements (especially graphs) - Handle PDF content as proper glm.Part with mime_type=application/pdf - Maintain the multimodal capabilities of Gemini for analyzing graphs and figures within PDFs **How will this feature improve your workflow / experience?** This feature will significantly streamline the workflow by: - Eliminating the need for separate processing of text and images from PDFs - Enabling direct analysis of graphs and figures within documents - Reducing the manual effort required to prepare PDF content for LLM processing - Making the Vertex AI provider as capable as Google AI Studio in terms of PDF handling ### 2. Additional context or comments - Special emphasis on graph recognition and interpretation capabilities - Technical implementation might involve: - Adding 'document' feature to gemini-*.yaml - Modifying llm.py to handle PDF documents as glm.Part - Ensuring proper MIME type handling for PDFs - Previous implementation attempts have been made but were unsuccessful in getting the PDF processing to work correctly ### 3. Can you help us with this feature? - [X] I am interested in contributing to this feature.
yindo added the 💪 enhancement label 2026-02-21 18:20:21 -05:00
yindo closed this issue 2026-02-21 18:20:22 -05:00
yindo commented 2026-02-21 18:20:22 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 25, 2025):

Hi, @wmoto-ai. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

  • Request for PDF processing support for Vertex AI Gemini provider.
  • Aims to replicate functionality from Google AI Studio.
  • Would enhance workflow by allowing direct PDF uploads and processing.
  • You expressed willingness to contribute to development.
  • No comments or developments have occurred since the issue was opened.

Next Steps:

  • Please confirm if this issue is still relevant to the latest version of the Dify repository by commenting here.
  • If there is no further activity, the issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Jan 25, 2025): Hi, @wmoto-ai. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary:** - Request for PDF processing support for Vertex AI Gemini provider. - Aims to replicate functionality from Google AI Studio. - Would enhance workflow by allowing direct PDF uploads and processing. - You expressed willingness to contribute to development. - No comments or developments have occurred since the issue was opened. **Next Steps:** - Please confirm if this issue is still relevant to the latest version of the Dify repository by commenting here. - If there is no further activity, the issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
yindo commented 2026-02-21 18:20:22 -05:00
Author
Owner
Copy Link
Copy Source

@wmoto-ai commented on GitHub (Feb 4, 2025):

Hi @dosubot, thanks for the follow-up. I've tested with the latest version (0.15.2) and can confirm that the PDF processing feature for Vertex AI Gemini provider is still missing. This is a critical feature for those of us who have to use Vertex AI for policy/security reasons. I'm still willing to help contribute if there's a clear direction from the team. Could we please keep this issue open for further discussion and updates? Thanks again for your time!

@wmoto-ai commented on GitHub (Feb 4, 2025): Hi @dosubot, thanks for the follow-up. I've tested with the latest version (0.15.2) and can confirm that the PDF processing feature for Vertex AI Gemini provider is still missing. This is a critical feature for those of us who have to use Vertex AI for policy/security reasons. I'm still willing to help contribute if there's a clear direction from the team. Could we please keep this issue open for further discussion and updates? Thanks again for your time!
yindo commented 2026-02-21 18:20:22 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Feb 4, 2025):

Hi @takatost, the user @wmoto-ai has confirmed that the PDF processing feature for the Vertex AI Gemini provider is still missing in the latest version (0.15.2) and is critical for their use case. Could you please assist them with this issue? Thank you!

@dosubot[bot] commented on GitHub (Feb 4, 2025): Hi @takatost, the user @wmoto-ai has confirmed that the PDF processing feature for the Vertex AI Gemini provider is still missing in the latest version (0.15.2) and is critical for their use case. Could you please assist them with this issue? Thank you!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 💪 enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#7387
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?