langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

The results returned by the API for obtaining knowledge base document segments do not correctly handle the "page" and "limit" parameters #12766

New Issue
Closed
opened 2026-02-21 19:08:57 -05:00 by yindo · 1 comment
yindo commented 2026-02-21 19:08:57 -05:00
Owner
Copy Link
Copy Source

Originally created by @1071969980 on GitHub (Apr 10, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.20

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

As the title suggests, v1/datasets/{dataset_id}/documents/{document_id}/segments
This API does not handle “page” and “limit” parameters correctly.

the relative source code :

class SegmentApi(DatasetApiResource):
    def get(self, tenant_id, dataset_id, document_id):
        """Get segments."""
        # check dataset
        dataset_id = str(dataset_id)
        tenant_id = str(tenant_id)
        page = request.args.get("page", default=1, type=int)
        limit = request.args.get("limit", default=20, type=int)
        dataset = db.session.query(Dataset).filter(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id).first()
        if not dataset:
            raise NotFound("Dataset not found.")
        # check document
        document_id = str(document_id)
        document = DocumentService.get_document(dataset.id, document_id)
        if not document:
            raise NotFound("Document not found.")
        # check embedding model setting
        if dataset.indexing_technique == "high_quality":
            try:
                model_manager = ModelManager()
                model_manager.get_model_instance(
                    tenant_id=current_user.current_tenant_id,
                    provider=dataset.embedding_model_provider,
                    model_type=ModelType.TEXT_EMBEDDING,
                    model=dataset.embedding_model,
                )
            except LLMBadRequestError:
                raise ProviderNotInitializeError(
                    "No Embedding Model available. Please configure a valid provider in the Settings -> Model Provider."
                )
            except ProviderTokenNotInitError as ex:
                raise ProviderNotInitializeError(ex.description)

        parser = reqparse.RequestParser()
        parser.add_argument("status", type=str, action="append", default=[], location="args")
        parser.add_argument("keyword", type=str, default=None, location="args")
        args = parser.parse_args()

        status_list = args["status"]
        keyword = args["keyword"]

        segments, total = SegmentService.get_segments(
            document_id=document_id,
            tenant_id=current_user.current_tenant_id,
            status_list=args["status"],
            keyword=args["keyword"],
        )

        response = {
            "data": marshal(segments, segment_fields),
            "doc_form": document.doc_form,
            "total": total,
            "has_more": len(segments) == limit,
            "limit": limit,
            "page": page,
        }

        return response, 200

✔️ Expected Behavior

Correctly returns the corresponding pagination and quantity limits

❌ Actual Behavior

Returns all segments in the document

Originally created by @1071969980 on GitHub (Apr 10, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.20 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce As the title suggests, v1/datasets/{dataset_id}/documents/{document_id}/segments This API does not handle “page” and “limit” parameters correctly. the relative source code : ``` python class SegmentApi(DatasetApiResource): def get(self, tenant_id, dataset_id, document_id): """Get segments.""" # check dataset dataset_id = str(dataset_id) tenant_id = str(tenant_id) page = request.args.get("page", default=1, type=int) limit = request.args.get("limit", default=20, type=int) dataset = db.session.query(Dataset).filter(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id).first() if not dataset: raise NotFound("Dataset not found.") # check document document_id = str(document_id) document = DocumentService.get_document(dataset.id, document_id) if not document: raise NotFound("Document not found.") # check embedding model setting if dataset.indexing_technique == "high_quality": try: model_manager = ModelManager() model_manager.get_model_instance( tenant_id=current_user.current_tenant_id, provider=dataset.embedding_model_provider, model_type=ModelType.TEXT_EMBEDDING, model=dataset.embedding_model, ) except LLMBadRequestError: raise ProviderNotInitializeError( "No Embedding Model available. Please configure a valid provider in the Settings -> Model Provider." ) except ProviderTokenNotInitError as ex: raise ProviderNotInitializeError(ex.description) parser = reqparse.RequestParser() parser.add_argument("status", type=str, action="append", default=[], location="args") parser.add_argument("keyword", type=str, default=None, location="args") args = parser.parse_args() status_list = args["status"] keyword = args["keyword"] segments, total = SegmentService.get_segments( document_id=document_id, tenant_id=current_user.current_tenant_id, status_list=args["status"], keyword=args["keyword"], ) response = { "data": marshal(segments, segment_fields), "doc_form": document.doc_form, "total": total, "has_more": len(segments) == limit, "limit": limit, "page": page, } return response, 200 ``` ### ✔️ Expected Behavior Correctly returns the corresponding pagination and quantity limits ### ❌ Actual Behavior Returns all segments in the document
yindo added the 🐞 bug label 2026-02-21 19:08:57 -05:00
yindo closed this issue 2026-02-21 19:08:57 -05:00
yindo commented 2026-02-21 19:08:57 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Apr 10, 2025):

Hello @1071969980! 👋 I'm Dosu, your friendly bot here to help with bugs, answer questions, and guide you on becoming a contributor. I'm here while you wait for a human maintainer. Let's tackle this together! 😊

The issue you're experiencing with the API not correctly handling the "page" and "limit" parameters seems to be related to how the SegmentService.get_segments method is being used. The current implementation does not appear to apply pagination directly within the get_segments method call, which is why all segments are being returned instead of paginated results.

To address this, you should ensure that the SegmentService.get_segments method is called with the page and limit parameters, and that it correctly implements pagination logic. Here's a general approach you might consider:

  1. Modify the get_segments Method: Ensure that the get_segments method in SegmentService accepts page and limit as parameters and uses them to slice the results accordingly.

  2. Implement Pagination Logic: Within the get_segments method, calculate the offset based on the page and limit, and use it to fetch only the required segments. For example:

    offset = (page - 1) * limit
segments = query.offset(offset).limit(limit).all()

  3. Update the API Response: Ensure that the response correctly reflects the paginated data, including the has_more field, which should indicate whether there are more segments to fetch.

By implementing these changes, the API should correctly handle the "page" and "limit" parameters, returning only the specified number of segments per page. If you need further assistance with the implementation details, reviewing the SegmentService.get_segments method and ensuring it aligns with these pagination principles would be a good next step.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (Apr 10, 2025): <!-- Greeting --> Hello @1071969980! 👋 I'm [Dosu](https://dosu.dev), your friendly bot here to help with bugs, answer questions, and guide you on becoming a contributor. I'm here while you wait for a human maintainer. Let's tackle this together! 😊 <!-- Answer --> The issue you're experiencing with the API not correctly handling the "page" and "limit" parameters seems to be related to how the `SegmentService.get_segments` method is being used. The current implementation does not appear to apply pagination directly within the `get_segments` method call, which is why all segments are being returned instead of paginated results. To address this, you should ensure that the `SegmentService.get_segments` method is called with the `page` and `limit` parameters, and that it correctly implements pagination logic. Here's a general approach you might consider: 1. **Modify the `get_segments` Method**: Ensure that the `get_segments` method in `SegmentService` accepts `page` and `limit` as parameters and uses them to slice the results accordingly. 2. **Implement Pagination Logic**: Within the `get_segments` method, calculate the offset based on the `page` and `limit`, and use it to fetch only the required segments. For example: ```python offset = (page - 1) * limit segments = query.offset(offset).limit(limit).all() ``` 3. **Update the API Response**: Ensure that the response correctly reflects the paginated data, including the `has_more` field, which should indicate whether there are more segments to fetch. By implementing these changes, the API should correctly handle the "page" and "limit" parameters, returning only the specified number of segments per page. If you need further assistance with the implementation details, reviewing the `SegmentService.get_segments` method and ensuring it aligns with these pagination principles would be a good next step. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/d7cc12c8-0624-432a-b314-68f2776d640a?feedback_type=other)</sup>
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#12766
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?