Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #184] Azure OpenAI Embed API says: "Too many inputs. The max number of inputs is 16." #117

New Issue
Closed
opened 2026-02-22 18:17:59 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:17:59 -05:00
Owner
Copy Link
Copy Source

Originally created by @dandinu on GitHub (Aug 10, 2023).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/184

Hi,

I am using this app with Azure OpenAI and it seems like when I try to vectorize a document that results in more than 16 chunks the upsert fails with this error:

SELECT * FROM workspace_documents WHERE workspaceId = 3
The namespace is docs
Adding new vectorized document into namespace docs
Chunks created from document: 26
addDocumentToNamespace {
  error: {
    message: 'Too many inputs. The max number of inputs is 16.  We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.',
    type: 'invalid_request_error',
    param: null,
    code: null
  }
}
Failed to vectorize custom-documents/comaround-test-924e5f4d-985f-4118-a33e-92d853a5c247.json
BEGIN TRANSACTION

Whenever I select a document that results in anything smaller it works just fine. Here is a working example (under 16 chunks):

SELECT * FROM workspace_documents WHERE workspaceId = 3
The namespace is docs
Adding new vectorized document into namespace docs
Chunks created from document: 3
After vectors.push the documentVectors is: [
  {
    docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18',
    vectorId: 'af19b5fe-8830-4303-9a42-212deb5493a0'
  }
]
After vectors.push the documentVectors is: [
  {
    docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18',
    vectorId: 'af19b5fe-8830-4303-9a42-212deb5493a0'
  },
  {
    docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18',
    vectorId: '96fa791b-2c50-42d8-b121-7fed39d36cee'
  }
]
After vectors.push the documentVectors is: [
  {
    docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18',
    vectorId: 'af19b5fe-8830-4303-9a42-212deb5493a0'
  },
  {
    docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18',
    vectorId: '96fa791b-2c50-42d8-b121-7fed39d36cee'
  },
  {
    docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18',
    vectorId: 'e63dfde7-8d39-422c-ac1c-e900110dd553'
  }
]
Inserting vectorized chunks into Pinecone.
Caching vectorized results of custom-documents/rubber-ducks-making-2eeaa089-6f7c-4ed7-beaf-8cb41decd529.json to prevent duplicated embedding.
BEGIN TRANSACTION

It seems to be related to #24, when the app moved to upload all the chunks in one single API call to improve performance. Is there any way to select the max batch size?

Originally created by @dandinu on GitHub (Aug 10, 2023). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/184 Hi, I am using this app with Azure OpenAI and it seems like when I try to vectorize a document that results in more than 16 chunks the upsert fails with this error: ``` SELECT * FROM workspace_documents WHERE workspaceId = 3 The namespace is docs Adding new vectorized document into namespace docs Chunks created from document: 26 addDocumentToNamespace { error: { message: 'Too many inputs. The max number of inputs is 16. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.', type: 'invalid_request_error', param: null, code: null } } Failed to vectorize custom-documents/comaround-test-924e5f4d-985f-4118-a33e-92d853a5c247.json BEGIN TRANSACTION ``` Whenever I select a document that results in anything smaller it works just fine. Here is a working example (under 16 chunks): ``` SELECT * FROM workspace_documents WHERE workspaceId = 3 The namespace is docs Adding new vectorized document into namespace docs Chunks created from document: 3 After vectors.push the documentVectors is: [ { docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18', vectorId: 'af19b5fe-8830-4303-9a42-212deb5493a0' } ] After vectors.push the documentVectors is: [ { docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18', vectorId: 'af19b5fe-8830-4303-9a42-212deb5493a0' }, { docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18', vectorId: '96fa791b-2c50-42d8-b121-7fed39d36cee' } ] After vectors.push the documentVectors is: [ { docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18', vectorId: 'af19b5fe-8830-4303-9a42-212deb5493a0' }, { docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18', vectorId: '96fa791b-2c50-42d8-b121-7fed39d36cee' }, { docId: 'a894474d-b3f8-4d4c-bcd5-d5bb22c18a18', vectorId: 'e63dfde7-8d39-422c-ac1c-e900110dd553' } ] Inserting vectorized chunks into Pinecone. Caching vectorized results of custom-documents/rubber-ducks-making-2eeaa089-6f7c-4ed7-beaf-8cb41decd529.json to prevent duplicated embedding. BEGIN TRANSACTION ``` It seems to be related to #24, when the app moved to upload all the chunks in one single API call to improve performance. Is there any way to select the max batch size?
yindo added the bug label 2026-02-22 18:17:59 -05:00
yindo closed this issue 2026-02-22 18:17:59 -05:00
yindo commented 2026-02-22 18:17:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 11, 2023):

This is weird, im wondering if this is a known limitation of Azure because I have never come across it on the normal openAI library 🤔

@timothycarambat commented on GitHub (Aug 11, 2023): This is weird, im wondering if this is a known limitation of Azure because I have never come across it on the normal openAI library 🤔
yindo commented 2026-02-22 18:17:59 -05:00
Author
Owner
Copy Link
Copy Source

@dandinu commented on GitHub (Aug 12, 2023):

I believe this is a known issue on Azure. Microsoft describes it better here: https://learn.microsoft.com/en-us/azure/ai-services/openai/faq#i-am-trying-to-use-embeddings-and-received-the-error--invalidrequesterror--too-many-inputs--the-max-number-of-inputs-is-1---how-do-i-fix-this-

I tried modifying server\utils\vectorDbProviders\pinecone\index.js:164 to limit the chunks from 100 to 16 but, weirdly, I still get the error:

        for (const chunk of toChunks(vectors, 16)) {
          chunks.push(chunk);
          await pineconeIndex.upsert({
            upsertRequest: {
              vectors: [...chunk],
              namespace,
            },
          });
        }

Is there any other place where I need to look to change this?

@dandinu commented on GitHub (Aug 12, 2023): I believe this is a known issue on Azure. Microsoft describes it better here: https://learn.microsoft.com/en-us/azure/ai-services/openai/faq#i-am-trying-to-use-embeddings-and-received-the-error--invalidrequesterror--too-many-inputs--the-max-number-of-inputs-is-1---how-do-i-fix-this- I tried modifying `server\utils\vectorDbProviders\pinecone\index.js:164` to limit the chunks from 100 to 16 but, weirdly, I still get the error: ``` for (const chunk of toChunks(vectors, 16)) { chunks.push(chunk); await pineconeIndex.upsert({ upsertRequest: { vectors: [...chunk], namespace, }, }); } ``` Is there any other place where I need to look to change this?
yindo commented 2026-02-22 18:17:59 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Aug 22, 2023):

@dandinu Just merged the fix for this. For posterity, the embeddings are done in the AI model (for now) as opposed to the vector database so the proper place to look was server/utils/AiProviders/azureOpenAi/index.js in the embedChunks function

@timothycarambat commented on GitHub (Aug 22, 2023): @dandinu Just merged the fix for this. For posterity, the embeddings are done in the AI model (for now) as opposed to the vector database so the proper place to look was `server/utils/AiProviders/azureOpenAi/index.js` in the `embedChunks` function
yindo commented 2026-02-22 18:17:59 -05:00
Author
Owner
Copy Link
Copy Source

@dandinu commented on GitHub (Aug 22, 2023):

Great work, now I see the commit with the changes. This helps!

@dandinu commented on GitHub (Aug 22, 2023): Great work, now I see the commit with the changes. This helps!
yindo changed title from Azure OpenAI Embed API says: "Too many inputs. The max number of inputs is 16." to [GH-ISSUE #184] Azure OpenAI Embed API says: "Too many inputs. The max number of inputs is 16." 2026-06-05 14:33:34 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#117
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?