langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

The chunk_overlap parameter is not documented in the Knowledge API specification. #7430

New Issue
Closed
opened 2026-02-21 18:20:33 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 18:20:33 -05:00
Owner
Copy Link
Copy Source

Originally created by @yagiyuki on GitHub (Dec 28, 2024).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Provide a description of requested docs changes

It is possible to specify "chunk_overlap" as a parameter for document creation in the Knowledge API.
However, this is not mentioned in the API specifications, so an addition is required.

ex: Create a Document from a Fileにおけるchunk_overlap

curl \
  --location \
  --request POST \
  "http://localhost/v1/datasets/${dataset_id}/document/create-by-file" \
  --header "Authorization: Bearer ${api_key}" \
  --form 'data={
    "indexing_technique": "high_quality",
    "process_rule": {
      "rules": {
        "pre_processing_rules": [
          { "id": "remove_extra_spaces", "enabled": true },
          { "id": "remove_urls_emails",  "enabled": true }
        ],
        "segmentation": {
          "separator": "###",
          "max_tokens": 500,
          "chunk_overlap": 100
        }
      },
      "mode": "custom"
    }
  };type=application/json' \
  --form 'file=@"test.txt"'
Originally created by @yagiyuki on GitHub (Dec 28, 2024). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Provide a description of requested docs changes It is possible to specify "chunk_overlap" as a parameter for document creation in the Knowledge API. However, this is not mentioned in the API specifications, so an addition is required. ex: Create a Document from a Fileにおけるchunk_overlap ```bash curl \ --location \ --request POST \ "http://localhost/v1/datasets/${dataset_id}/document/create-by-file" \ --header "Authorization: Bearer ${api_key}" \ --form 'data={ "indexing_technique": "high_quality", "process_rule": { "rules": { "pre_processing_rules": [ { "id": "remove_extra_spaces", "enabled": true }, { "id": "remove_urls_emails", "enabled": true } ], "segmentation": { "separator": "###", "max_tokens": 500, "chunk_overlap": 100 } }, "mode": "custom" } };type=application/json' \ --form 'file=@"test.txt"' ```
yindo added the 📚 documentation label 2026-02-21 18:20:33 -05:00
yindo closed this issue 2026-02-21 18:20:33 -05:00
yindo commented 2026-02-21 18:20:33 -05:00
Author
Owner
Copy Link
Copy Source

@yagiyuki commented on GitHub (Dec 30, 2024):

target or modification

  • Create a Document from Text
    • Modifying the sample request : unnecessary
  • Create a Document from a File
    • Modifying the sample request : necessary
  • Update a Document with Text
    • Modifying the sample request : unnecessary
  • Update a Document with a File
    • Modifying the sample request : necessary
@yagiyuki commented on GitHub (Dec 30, 2024): target or modification * Create a Document from Text * Modifying the sample request : unnecessary * Create a Document from a File * Modifying the sample request : necessary * Update a Document with Text * Modifying the sample request : unnecessary * Update a Document with a File * Modifying the sample request : necessary
yindo commented 2026-02-21 18:20:33 -05:00
Author
Owner
Copy Link
Copy Source

@yagiyuki commented on GitHub (Dec 30, 2024):

  • Create a Document from Text
curl \
  --location \
  --request POST \
  "http://localhost/v1/datasets/${dataset_id}/document/create-by-text" \
  --header "Authorization: Bearer ${api_key}" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "create-by-text",
    "text": "create-by-text",
    "indexing_technique": "high_quality",
    "process_rule": {
      "rules": {
        "pre_processing_rules": [
          { "id": "remove_extra_spaces", "enabled": true },
          { "id": "remove_urls_emails",  "enabled": true }
        ],
        "segmentation": {
          "separator": "###",
          "max_tokens": 300,
          "chunk_overlap": 30
        }
      },
      "mode": "custom"
    }
  }'
  • Create a Document from a File
curl \
  --location \
  --request POST \
  "http://localhost/v1/datasets/${dataset_id}/document/create-by-file" \
  --header "Authorization: Bearer ${api_key}" \
  --form 'data={
    "indexing_technique": "high_quality",
    "process_rule": {
      "rules": {
        "pre_processing_rules": [
          { "id": "remove_extra_spaces", "enabled": true },
          { "id": "remove_urls_emails",  "enabled": true }
        ],
        "segmentation": {
          "separator": "###",
          "max_tokens": 300,
          "chunk_overlap": 30
        }
      },
      "mode": "custom"
    }
  };type=application/json' \
  --form 'file=@"test.txt"'
  • Update a Document with Text
curl \
  --location \
  --request POST \
  "http://localhost/v1/datasets/${dataset_id}/documents/${document_id}/update-by-text" \
  --header "Authorization: Bearer ${api_key}" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "create-by-text",
    "text": "create-by-text",
    "indexing_technique": "high_quality",
    "process_rule": {
      "rules": {
        "pre_processing_rules": [
          { "id": "remove_extra_spaces", "enabled": true },
          { "id": "remove_urls_emails",  "enabled": true }
        ],
        "segmentation": {
          "separator": "###",
          "max_tokens": 300,
          "chunk_overlap": 50
        }
      },
      "mode": "custom"
    }
  }'
  • Update a Document with a File
curl \
  --location \
  --request POST \
  "http://localhost/v1/datasets/${dataset_id}/documents/${document_id}/update-by-file" \
  --header "Authorization: Bearer ${api_key}" \
  --form 'data={
    "indexing_technique": "high_quality",
    "process_rule": {
      "rules": {
        "pre_processing_rules": [
          { "id": "remove_extra_spaces", "enabled": true },
          { "id": "remove_urls_emails",  "enabled": true }
        ],
        "segmentation": {
          "separator": "###",
          "max_tokens": 300,
          "chunk_overlap": 55
        }
      },
      "mode": "custom"
    }
  }' \
  --form "file=@test.txt"
@yagiyuki commented on GitHub (Dec 30, 2024): * Create a Document from Text ``` curl \ --location \ --request POST \ "http://localhost/v1/datasets/${dataset_id}/document/create-by-text" \ --header "Authorization: Bearer ${api_key}" \ --header "Content-Type: application/json" \ --data '{ "name": "create-by-text", "text": "create-by-text", "indexing_technique": "high_quality", "process_rule": { "rules": { "pre_processing_rules": [ { "id": "remove_extra_spaces", "enabled": true }, { "id": "remove_urls_emails", "enabled": true } ], "segmentation": { "separator": "###", "max_tokens": 300, "chunk_overlap": 30 } }, "mode": "custom" } }' ``` * Create a Document from a File ``` curl \ --location \ --request POST \ "http://localhost/v1/datasets/${dataset_id}/document/create-by-file" \ --header "Authorization: Bearer ${api_key}" \ --form 'data={ "indexing_technique": "high_quality", "process_rule": { "rules": { "pre_processing_rules": [ { "id": "remove_extra_spaces", "enabled": true }, { "id": "remove_urls_emails", "enabled": true } ], "segmentation": { "separator": "###", "max_tokens": 300, "chunk_overlap": 30 } }, "mode": "custom" } };type=application/json' \ --form 'file=@"test.txt"' ``` * Update a Document with Text ``` curl \ --location \ --request POST \ "http://localhost/v1/datasets/${dataset_id}/documents/${document_id}/update-by-text" \ --header "Authorization: Bearer ${api_key}" \ --header "Content-Type: application/json" \ --data '{ "name": "create-by-text", "text": "create-by-text", "indexing_technique": "high_quality", "process_rule": { "rules": { "pre_processing_rules": [ { "id": "remove_extra_spaces", "enabled": true }, { "id": "remove_urls_emails", "enabled": true } ], "segmentation": { "separator": "###", "max_tokens": 300, "chunk_overlap": 50 } }, "mode": "custom" } }' ``` * Update a Document with a File ``` curl \ --location \ --request POST \ "http://localhost/v1/datasets/${dataset_id}/documents/${document_id}/update-by-file" \ --header "Authorization: Bearer ${api_key}" \ --form 'data={ "indexing_technique": "high_quality", "process_rule": { "rules": { "pre_processing_rules": [ { "id": "remove_extra_spaces", "enabled": true }, { "id": "remove_urls_emails", "enabled": true } ], "segmentation": { "separator": "###", "max_tokens": 300, "chunk_overlap": 55 } }, "mode": "custom" } }' \ --form "file=@test.txt" ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 📚 documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#7430
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?