Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4032] [BUG]: TypeError: tokenizer is not a function #2564

New Issue
Closed
opened 2026-02-22 18:30:16 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:30:16 -05:00
Owner
Copy Link
Copy Source

Originally created by @noelmateng on GitHub (Jun 23, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4032

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

Getting this error when running multiple queries concurrently.

2025-06-23 11:05:29 [backend] error: TypeError: tokenizer is not a function
2025-06-23 11:05:29     at NativeEmbeddingReranker.rerank (/app/server/utils/EmbeddingRerankers/native/index.js:215:20)
2025-06-23 11:05:29     at async Object.rerankedSimilarityResponse (/app/server/utils/vectorDbProviders/lance/index.js:114:5)
2025-06-23 11:05:29     at async Object.performSimilaritySearch (/app/server/utils/vectorDbProviders/lance/index.js:405:9)
2025-06-23 11:05:29     at async Object.chatSync (/app/server/utils/chats/apiChatHandler.js:213:9)
2025-06-23 11:05:29     at async /app/server/endpoints/api/workspace/index.js:681:24
2025-06-23 11:05:29 [backend] error: LanceDB::rerankedSimilarityResponse tokenizer is not a function

This results in the sources being returned as empty array.

Based on my analysis,

The issue seems to be due to this assignment here.
I think when we run multiple queries concurrently, the NativeEmbeddingReranker.#transformers gets set in the first query (api call) while the second query exits from here

This causes the tokenizer here to be undefined here
and we get the error: TypeError: tokenizer is not a function

Are there known steps to reproduce?

  1. Run anything labs locally on docker

  2. Use default system embedding model

  3. Load some documents into your workspace

  4. Run multiple chats in query mode in parallel. My concurrency was 2.

Originally created by @noelmateng on GitHub (Jun 23, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4032 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? Getting this error when running multiple queries concurrently. ``` 2025-06-23 11:05:29 [backend] error: TypeError: tokenizer is not a function 2025-06-23 11:05:29 at NativeEmbeddingReranker.rerank (/app/server/utils/EmbeddingRerankers/native/index.js:215:20) 2025-06-23 11:05:29 at async Object.rerankedSimilarityResponse (/app/server/utils/vectorDbProviders/lance/index.js:114:5) 2025-06-23 11:05:29 at async Object.performSimilaritySearch (/app/server/utils/vectorDbProviders/lance/index.js:405:9) 2025-06-23 11:05:29 at async Object.chatSync (/app/server/utils/chats/apiChatHandler.js:213:9) 2025-06-23 11:05:29 at async /app/server/endpoints/api/workspace/index.js:681:24 2025-06-23 11:05:29 [backend] error: LanceDB::rerankedSimilarityResponse tokenizer is not a function ``` This results in the sources being returned as empty array. Based on my analysis, The issue seems to be due to this [assignment here](https://github.com/Mintplex-Labs/anything-llm/blob/c4f49ff6b445f8f1b15d3bd42f8a2d24317b745c/server/utils/EmbeddingRerankers/native/index.js#L82). I think when we run multiple queries concurrently, the NativeEmbeddingReranker.#transformers gets set in the first query (api call) while the second query exits from [here](https://github.com/Mintplex-Labs/anything-llm/blob/c4f49ff6b445f8f1b15d3bd42f8a2d24317b745c/server/utils/EmbeddingRerankers/native/index.js#L74) This causes the tokenizer here to be undefined [here](https://github.com/Mintplex-Labs/anything-llm/blob/c4f49ff6b445f8f1b15d3bd42f8a2d24317b745c/server/utils/EmbeddingRerankers/native/index.js#L211) and we get the error: TypeError: tokenizer is not a function ### Are there known steps to reproduce? 1. Run anything labs locally on docker 2. Use default system embedding model 3. Load some documents into your workspace 4. Run multiple chats in query mode in parallel. My concurrency was 2.
yindo added the possible bugneeds info / can't replicateinvestigating labels 2026-02-22 18:30:16 -05:00
yindo closed this issue 2026-02-22 18:30:16 -05:00
yindo commented 2026-02-22 18:30:16 -05:00
Author
Owner
Copy Link
Copy Source

@anytrace-ai-support-engineer-yc-s25 commented on GitHub (Jun 29, 2025):

Summary of the issue

The error is caused by a race condition in lazy singleton initialization.
This happens because #transformers cache flag is written before the awaited module resolves, so concurrent requests see a half-initialised cache.

Next steps for @noelmateng

None

Next steps for Multiplex-Labs

Address the race condition.
Consider wrapping the whole initialization in a promise that every call awaits:

class NativeEmbeddingReranker {
  static #initPromise;
  static #model;
  static #tokenizer;

  async initClient() {
    if (!NativeEmbeddingReranker.#initPromise) {
      NativeEmbeddingReranker.#initPromise = (async () => {
        const { pipeline, AutoTokenizer } = await import("@xenova/transformers");

        // load once
        NativeEmbeddingReranker.#tokenizer = await AutoTokenizer.from_pretrained(
          "Xenova/bge-reranker-base"
        );
        NativeEmbeddingReranker.#model = await pipeline(
          "text-classification",
          "Xenova/bge-reranker-base"
        );
      })();
    }
    await NativeEmbeddingReranker.#initPromise;   // every caller waits here
  }
}

@Mintplex-Labs @timothycarambat

If you'd like Anytrace (YC S25) to keep investigating your support tickets, reach out to taira@anytrace.ai to discuss how we can keep supporting your team.

@anytrace-ai-support-engineer-yc-s25 commented on GitHub (Jun 29, 2025): ### Summary of the issue The error is caused by a race condition in lazy singleton initialization. This happens because [`#transformers` cache flag is written](https://github.com/Mintplex-Labs/anything-llm/blob/c4f49ff6b445f8f1b15d3bd42f8a2d24317b745c/server/utils/EmbeddingRerankers/native/index.js#L82) before the [awaited module resolves](https://github.com/Mintplex-Labs/anything-llm/blob/c4f49ff6b445f8f1b15d3bd42f8a2d24317b745c/server/utils/EmbeddingRerankers/native/index.js#L91-L92), so concurrent requests see a half-initialised cache. ### Next steps for @noelmateng None ### Next steps for Multiplex-Labs Address the race condition. Consider wrapping the whole initialization in a promise that every call awaits: ``` class NativeEmbeddingReranker { static #initPromise; static #model; static #tokenizer; async initClient() { if (!NativeEmbeddingReranker.#initPromise) { NativeEmbeddingReranker.#initPromise = (async () => { const { pipeline, AutoTokenizer } = await import("@xenova/transformers"); // load once NativeEmbeddingReranker.#tokenizer = await AutoTokenizer.from_pretrained( "Xenova/bge-reranker-base" ); NativeEmbeddingReranker.#model = await pipeline( "text-classification", "Xenova/bge-reranker-base" ); })(); } await NativeEmbeddingReranker.#initPromise; // every caller waits here } } ``` ### @Mintplex-Labs @timothycarambat If you'd like Anytrace (YC S25) to keep investigating your support tickets, reach out to taira@anytrace.ai to discuss how we can keep supporting your team.
yindo commented 2026-02-22 18:30:16 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jun 30, 2025):

Hi @noelmateng, thanks for reporting this! Could you give me some more details on how to replicate this issue?

I have tried in both the latest docker image and cloning the github repo and running everything locally and was unable to replicate it.

I tried running 2 concurrent chats on a workspace via the developer API and also just through the normal chat UI in 2 tabs and was unable to run into the same issue you're experiencing here. If you could please give me some more info here it would be greatly appreciated!

@shatfield4 commented on GitHub (Jun 30, 2025): Hi @noelmateng, thanks for reporting this! Could you give me some more details on how to replicate this issue? I have tried in both the latest docker image and cloning the github repo and running everything locally and was unable to replicate it. I tried running 2 concurrent chats on a workspace via the developer API and also just through the normal chat UI in 2 tabs and was unable to run into the same issue you're experiencing here. If you could please give me some more info here it would be greatly appreciated!
yindo commented 2026-02-22 18:30:16 -05:00
Author
Owner
Copy Link
Copy Source

@noelmateng commented on GitHub (Jul 1, 2025):

Hi @shatfield4,
what i found was once you have started instance in docker, run multiple workspace chats using this endpoint: "/api/v1/workspace/{slug}/chat",
If you are working with js/ts, ensure you call it in something like promise.all/promise.allSettled.
You'll only be able to reproduce this when the instance has just started and the tokenizers are not yet loaded.

My workspace settings:

{'id': 2,
 'name': '<redacted>',
 'slug': '<redacted>',
 'vectorTag': None,
 'createdAt': '2025-06-22T14:49:50.166Z',
 'openAiTemp': 0.7,
 'openAiHistory': 20,
 'lastUpdatedAt': '2025-06-22T14:49:50.166Z',
 'openAiPrompt': None,
 'similarityThreshold': 0.5,
 'chatProvider': None,
 'chatModel': None,
 'topN': 4,
 'chatMode': 'query',
 'pfpFilename': None,
 'agentProvider': None,
 'agentModel': None,
 'queryRefusalResponse': None,
 'vectorSearchMode': 'rerank',
 'threads': []}
@noelmateng commented on GitHub (Jul 1, 2025): Hi @shatfield4, what i found was once you have started instance in docker, run multiple workspace chats using this endpoint: "/api/v1/workspace/{slug}/chat", If you are working with js/ts, ensure you call it in something like promise.all/promise.allSettled. You'll only be able to reproduce this when the instance has just started and the tokenizers are not yet loaded. My workspace settings: ``` {'id': 2, 'name': '<redacted>', 'slug': '<redacted>', 'vectorTag': None, 'createdAt': '2025-06-22T14:49:50.166Z', 'openAiTemp': 0.7, 'openAiHistory': 20, 'lastUpdatedAt': '2025-06-22T14:49:50.166Z', 'openAiPrompt': None, 'similarityThreshold': 0.5, 'chatProvider': None, 'chatModel': None, 'topN': 4, 'chatMode': 'query', 'pfpFilename': None, 'agentProvider': None, 'agentModel': None, 'queryRefusalResponse': None, 'vectorSearchMode': 'rerank', 'threads': []} ```
yindo changed title from [BUG]: TypeError: tokenizer is not a function to [GH-ISSUE #4032] [BUG]: TypeError: tokenizer is not a function 2026-06-05 14:47:17 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2564
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?