Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4753] [BUG]: 1.9.1 pull request 4121 - addition of paperless-ngx connector. Paperless-ngx connector only fetches first page of documents (pagination not implemented) #3001

New Issue
Closed
opened 2026-02-22 18:32:12 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:32:12 -05:00
Owner
Copy Link
Copy Source

Originally created by @jhsmith409 on GitHub (Dec 10, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4753

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

Paperless-ngx connector only fetches first page of documents (pagination not implemented)

Description

The new Paperless-ngx data connector introduced in v1.9.1 (https://github.com/Mintplex-Labs/anything-llm/pull/4121) only retrieves approximately 25 documents, even when the Paperless-ngx instance contains thousands of documents. This appears to be because the connector fetches /api/documents/ without handling pagination.

Steps to Reproduce

Configure the Paperless-ngx data connector with a valid instance URL and API token
Connect to a Paperless-ngx instance that contains more than 25 documents
Observe that only ~20-25 documents are available for import
Expected Behavior

All documents from the Paperless-ngx instance should be available for import, regardless of the total count.

Actual Behavior

Only the first page of results (~25 documents) is returned. Documents beyond the first page are not fetched.

Root Cause

The Paperless-ngx API returns paginated results by default. The response includes:

count: total number of documents
next: URL to fetch the next page
previous: URL to fetch the previous page
results: array of documents for the current page
The connector appears to only process results from the initial request without following the next URL to retrieve subsequent pages.

Suggested Fix

The fetchAllDocuments() method should iterate through all pages until next is null. Alternatively, use the page_size parameter to request more documents per page: /api/documents/?page_size=1000

Environment

AnythingLLM version: 1.9.1
Paperless-ngx version: (any)
Deployment: Docker

Are there known steps to reproduce?

No response

Originally created by @jhsmith409 on GitHub (Dec 10, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4753 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? Paperless-ngx connector only fetches first page of documents (pagination not implemented) Description The new Paperless-ngx data connector introduced in v1.9.1 (https://github.com/Mintplex-Labs/anything-llm/pull/4121) only retrieves approximately 25 documents, even when the Paperless-ngx instance contains thousands of documents. This appears to be because the connector fetches /api/documents/ without handling pagination. Steps to Reproduce Configure the Paperless-ngx data connector with a valid instance URL and API token Connect to a Paperless-ngx instance that contains more than 25 documents Observe that only ~20-25 documents are available for import Expected Behavior All documents from the Paperless-ngx instance should be available for import, regardless of the total count. Actual Behavior Only the first page of results (~25 documents) is returned. Documents beyond the first page are not fetched. Root Cause The Paperless-ngx API returns paginated results by default. The response includes: count: total number of documents next: URL to fetch the next page previous: URL to fetch the previous page results: array of documents for the current page The connector appears to only process results from the initial request without following the next URL to retrieve subsequent pages. Suggested Fix The fetchAllDocuments() method should iterate through all pages until next is null. Alternatively, use the page_size parameter to request more documents per page: /api/documents/?page_size=1000 Environment AnythingLLM version: 1.9.1 Paperless-ngx version: (any) Deployment: Docker ### Are there known steps to reproduce? _No response_
yindo added the possible buginvestigating labels 2026-02-22 18:32:13 -05:00
yindo closed this issue 2026-02-22 18:32:13 -05:00
yindo changed title from [BUG]: 1.9.1 pull request 4121 - addition of paperless-ngx connector. Paperless-ngx connector only fetches first page of documents (pagination not implemented) to [GH-ISSUE #4753] [BUG]: 1.9.1 pull request 4121 - addition of paperless-ngx connector. Paperless-ngx connector only fetches first page of documents (pagination not implemented) 2026-06-05 14:49:45 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3001
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?