[GH-ISSUE #4105] [BUG]: BigInt Serialization Error with OpenRouter LLMs causing chat failures #2615

Closed
opened 2026-02-22 18:30:28 -05:00 by yindo · 4 comments
Owner

Originally created by @drockthedoc on GitHub (Jul 8, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4105

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

BigInt Serialization Error in AnythingLLM with OpenRouter LLMs

Bug Description

AnythingLLM crashes with BigInt serialization errors when using certain LLM providers (particularly OpenRouter with models like deepseek/deepseek-r1-distill-llama-70b:free) due to multiple JSON.stringify() calls throughout the codebase that cannot handle BigInt values.

Error Messages

[backend] error: Do not know how to serialize a BigInt
[backend] error: TypeError: Cannot read properties of null (reading 'id')
    at streamChatWithWorkspace (/app/server/utils/chats/stream.js:278:20)

Environment

  • AnythingLLM Version: Latest Docker (tested on commit 14fa079)
  • Deployment: Docker container
  • LLM Provider: OpenRouter
  • Model: deepseek/deepseek-r1-distill-llama-70b:free
  • Vector Database: LanceDB
  • Embedder: Native embedder

Root Cause

The issue occurs because several parts of the codebase use JSON.stringify() without handling BigInt values properly. When LLM responses contain BigInt values (common with token counts, timestamps, or other numeric metadata), these calls fail and prevent chat creation, leading to null chat objects and subsequent errors.

Affected Files

The following files contain JSON.stringify() calls that need BigInt handling:

  1. /app/server/models/workspaceChats.js - Critical: Chat creation fails here
  2. /app/server/models/embedChats.js - Embed chat responses
  3. /app/server/utils/logger/index.js - Logger formatting
  4. /app/server/utils/helpers/chat/responses.js - Chat response handling
  5. /app/server/utils/chats/openaiCompatible.js - OpenAI compatibility layer
  6. /app/server/utils/helpers/chat/convertTo.js - Chat conversion utilities
  7. /app/server/utils/AiProviders/openRouter/index.js - OpenRouter provider
  8. /app/server/utils/chats/stream.js - Null safety for chat.id

Solution

Replace all instances of JSON.stringify(obj) with JSON.stringify(obj, (key, value) => typeof value === "bigint" ? value.toString() : value) to safely convert BigInt values to strings during serialization.

Specific Fixes Applied

1. Fix Chat Creation (Critical)

File: /app/server/models/workspaceChats.js

// Before (line ~18)
response: JSON.stringify(response),

// After
response: JSON.stringify(response, (key, value) => typeof value === "bigint" ? value.toString() : value),

2. Fix Embed Chats

File: /app/server/models/embedChats.js

// Fix multiple instances (lines ~28, ~29, ~51)
response: JSON.stringify(response, (key, value) => typeof value === "bigint" ? value.toString() : value),
connection_information: JSON.stringify(connection_information, (key, value) => typeof value === "bigint" ? value.toString() : value),
response: JSON.stringify(responseRest, (key, value) => typeof value === "bigint" ? value.toString() : value)

3. Fix Logger

File: /app/server/utils/logger/index.js

// Before (line ~37)
return JSON.stringify(arg);

// After
return JSON.stringify(arg, (key, value) => typeof value === "bigint" ? value.toString() : value);

4. Fix Stream Null Safety

File: /app/server/utils/chats/stream.js

// Before (line ~278)
chatId: chat.id,

// After
chatId: chat?.id || null,

5. Fix Other Response Handlers

Similar fixes needed in:

  • /app/server/utils/helpers/chat/responses.js
  • /app/server/utils/chats/openaiCompatible.js
  • /app/server/utils/helpers/chat/convertTo.js
  • /app/server/utils/AiProviders/openRouter/index.js

Testing

After applying these fixes:

  • AnythingLLM starts without errors
  • Chat creation works with OpenRouter LLMs
  • Streaming responses function correctly
  • No BigInt serialization errors in logs
  • Telemetry shows successful sent_chat events

Impact

This bug prevents AnythingLLM from working with several popular LLM providers that return BigInt values in their responses, particularly:

  • OpenRouter models
  • Various open-source models through compatibility layers
  • Any provider returning token counts or timestamps as BigInt

Reproduction Steps

  1. Set up AnythingLLM with OpenRouter as LLM provider
  2. Configure a model like deepseek/deepseek-r1-distill-llama-70b:free
  3. Attempt to send a chat message
  4. Observe BigInt serialization errors and chat creation failures

Suggested Implementation

Consider creating a utility function for BigInt-safe JSON serialization:

// utils/json.js
function safeJsonStringify(obj, ...args) {
  return JSON.stringify(obj, (key, value) => typeof value === "bigint" ? value.toString() : value, ...args);
}

module.exports = { safeJsonStringify };

Then replace all JSON.stringify() calls with safeJsonStringify() throughout the codebase.

Priority

High - This bug prevents AnythingLLM from functioning with popular LLM providers and causes complete chat failure rather than graceful degradation.

Are there known steps to reproduce?

No response

Originally created by @drockthedoc on GitHub (Jul 8, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4105 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? # BigInt Serialization Error in AnythingLLM with OpenRouter LLMs ## Bug Description AnythingLLM crashes with BigInt serialization errors when using certain LLM providers (particularly OpenRouter with models like `deepseek/deepseek-r1-distill-llama-70b:free`) due to multiple `JSON.stringify()` calls throughout the codebase that cannot handle BigInt values. ## Error Messages ``` [backend] error: Do not know how to serialize a BigInt [backend] error: TypeError: Cannot read properties of null (reading 'id') at streamChatWithWorkspace (/app/server/utils/chats/stream.js:278:20) ``` ## Environment - **AnythingLLM Version**: Latest Docker (tested on commit 14fa079) - **Deployment**: Docker container - **LLM Provider**: OpenRouter - **Model**: `deepseek/deepseek-r1-distill-llama-70b:free` - **Vector Database**: LanceDB - **Embedder**: Native embedder ## Root Cause The issue occurs because several parts of the codebase use `JSON.stringify()` without handling BigInt values properly. When LLM responses contain BigInt values (common with token counts, timestamps, or other numeric metadata), these calls fail and prevent chat creation, leading to null chat objects and subsequent errors. ## Affected Files The following files contain `JSON.stringify()` calls that need BigInt handling: 1. `/app/server/models/workspaceChats.js` - **Critical**: Chat creation fails here 2. `/app/server/models/embedChats.js` - Embed chat responses 3. `/app/server/utils/logger/index.js` - Logger formatting 4. `/app/server/utils/helpers/chat/responses.js` - Chat response handling 5. `/app/server/utils/chats/openaiCompatible.js` - OpenAI compatibility layer 6. `/app/server/utils/helpers/chat/convertTo.js` - Chat conversion utilities 7. `/app/server/utils/AiProviders/openRouter/index.js` - OpenRouter provider 8. `/app/server/utils/chats/stream.js` - Null safety for chat.id ## Solution Replace all instances of `JSON.stringify(obj)` with `JSON.stringify(obj, (key, value) => typeof value === "bigint" ? value.toString() : value)` to safely convert BigInt values to strings during serialization. ### Specific Fixes Applied #### 1. Fix Chat Creation (Critical) **File**: `/app/server/models/workspaceChats.js` ```javascript // Before (line ~18) response: JSON.stringify(response), // After response: JSON.stringify(response, (key, value) => typeof value === "bigint" ? value.toString() : value), ``` #### 2. Fix Embed Chats **File**: `/app/server/models/embedChats.js` ```javascript // Fix multiple instances (lines ~28, ~29, ~51) response: JSON.stringify(response, (key, value) => typeof value === "bigint" ? value.toString() : value), connection_information: JSON.stringify(connection_information, (key, value) => typeof value === "bigint" ? value.toString() : value), response: JSON.stringify(responseRest, (key, value) => typeof value === "bigint" ? value.toString() : value) ``` #### 3. Fix Logger **File**: `/app/server/utils/logger/index.js` ```javascript // Before (line ~37) return JSON.stringify(arg); // After return JSON.stringify(arg, (key, value) => typeof value === "bigint" ? value.toString() : value); ``` #### 4. Fix Stream Null Safety **File**: `/app/server/utils/chats/stream.js` ```javascript // Before (line ~278) chatId: chat.id, // After chatId: chat?.id || null, ``` #### 5. Fix Other Response Handlers Similar fixes needed in: - `/app/server/utils/helpers/chat/responses.js` - `/app/server/utils/chats/openaiCompatible.js` - `/app/server/utils/helpers/chat/convertTo.js` - `/app/server/utils/AiProviders/openRouter/index.js` ## Testing After applying these fixes: - ✅ AnythingLLM starts without errors - ✅ Chat creation works with OpenRouter LLMs - ✅ Streaming responses function correctly - ✅ No BigInt serialization errors in logs - ✅ Telemetry shows successful `sent_chat` events ## Impact This bug prevents AnythingLLM from working with several popular LLM providers that return BigInt values in their responses, particularly: - OpenRouter models - Various open-source models through compatibility layers - Any provider returning token counts or timestamps as BigInt ## Reproduction Steps 1. Set up AnythingLLM with OpenRouter as LLM provider 2. Configure a model like `deepseek/deepseek-r1-distill-llama-70b:free` 3. Attempt to send a chat message 4. Observe BigInt serialization errors and chat creation failures ## Suggested Implementation Consider creating a utility function for BigInt-safe JSON serialization: ```javascript // utils/json.js function safeJsonStringify(obj, ...args) { return JSON.stringify(obj, (key, value) => typeof value === "bigint" ? value.toString() : value, ...args); } module.exports = { safeJsonStringify }; ``` Then replace all `JSON.stringify()` calls with `safeJsonStringify()` throughout the codebase. ## Priority **High** - This bug prevents AnythingLLM from functioning with popular LLM providers and causes complete chat failure rather than graceful degradation. ### Are there known steps to reproduce? _No response_
yindo added the needs info / can't replicate label 2026-02-22 18:30:28 -05:00
yindo closed this issue 2026-02-22 18:30:28 -05:00
Author
Owner

@shatfield4 commented on GitHub (Jul 8, 2025):

I'm unable to replicate this bug using the OpenRouter provider and the same deepseek model you said you used here. Is there any specific message you sent to get this error to replicate consistently?

@shatfield4 commented on GitHub (Jul 8, 2025): I'm unable to replicate this bug using the OpenRouter provider and the same deepseek model you said you used here. Is there any specific message you sent to get this error to replicate consistently?
Author
Owner

@drockthedoc commented on GitHub (Jul 8, 2025):

Um... i'm using my self-patched version of anythingllm at the moment, and i've changed things iteratively to refine the original prompt since I self-patched, but i'll give you the broad strokes:

here's my current system prompt for my workspace -- the one that was giving me problems may have differed a bit but was substantially similar (please don't judge me):


You are an expert in healthcare administration and psychometrics. Your task is to write one high-quality, multiple-choice question (MCQ) for a professional certification exam that tests higher-order thinking.

Primary Goal:
Generate a single MCQ that requires the test-taker to first, analyze a scenario containing specific data points to diagnose a problem, and second, apply a formal standard or regulation to determine the single, factually correct response.

Core Psychometric Principle: Test Diagnosis and Application
The question must be structured to differentiate candidates who can accurately interpret operational data from those who cannot. The thinking process is a two-step sequence: data analysis, then rule application.

How to Construct the Question Stem:

Present a Data-Driven Scenario: The stem MUST describe a realistic administrative or operational challenge by providing 2-3 specific, quantitative or qualitative data points (e.g., performance metrics, survey results, incident reports, budget figures, staff complaints).

Imply a Core Problem: These data points, when synthesized, must point to a clear underlying problem (e.g., a patient safety decline, a compliance gap, a workforce issue).

Ask for a Required Action: The question must ask for a specific, required action or process that follows from a correct diagnosis of the data, based on an explicit standard, law, or regulation.

Example of the Principle (Transforming a Type I to a Type II Question):

AVOID (Type I - Simple Recall): "According to the WHO, which strategy is recommended to mitigate burnout?"

This only tests knowledge of a fact.

EMPHASIZE (Type II - Analysis + Application): "A hospital's annual report reveals a 20% increase in nursing turnover, a significant rise in medication errors, and survey data showing widespread emotional exhaustion. Citing the WHO's definition of burnout as an occupational phenomenon, the hospital's strategic plan must primarily focus on which area?"

This forces the test-taker to first diagnose the problem from the data, then apply the WHO principle to find the correct answer.

Exam Blueprint:
[The blueprint list remains the same and would be inserted here]

Required Output Format:
Generate the MCQ using the exact structure below, with no additional commentary.

QUESTION STEM:
[Write a scenario containing 2-3 specific data points that imply a core problem. End with a direct question asking for a required action or process based on a specific standard or regulation.]

ANSWER OPTIONS:
[Provide four plausible options (A-D). They must be concise (1-7 words) and grammatically parallel. Foils should be plausible actions if one misinterprets the data or the governing standard.]
A. [Plausible but incorrect option]
B. [Plausible but incorrect option]
C. [The single correct option]
D. [Plausible but incorrect option]

CORRECT ANSWER:
[Provide only the letter of the correct answer.]

TESTING POINT:
[State the two-part skill being tested. Format as: "Can the candidate analyze [summary of data] and apply [the specific rule or standard] to determine the required action?"]

COGNITIVE LEVEL:
Type II

BLUEPRINT DOMAIN:
[List the full code and label for the chosen subdomain.]

REFERENCE:
[Provide a direct quote from an authoritative source (e.g., a law, regulatory body, or national standard) that provides the factual basis for the right answer.]

Blueprint Domains:
6.0 Human Resource Management and Workforce Development
6.1 Compensation and benefits practices
6.2 Conflicts and dualities of interest (e.g., industry relationships)
6.3 Conflict resolution and grievance procedures
6.4 Diversity, inclusion, and equity strategies
6.5 Employee safety, security, and health issues (e.g., OSHA, workplace violence)
6.6 Employee satisfaction assessment, engagement, motivation, and career development tools
6.7 Labor relations and laws (e.g., FMLA, FLSA, EEOC, ERISA, worker compensation)
6.8 Performance management systems (e.g., performance-based evaluation, rewards systems, disciplinary policies, and procedures)
6.9 Physician satisfaction assessment and engagement tools and techniques
6.10 Recruitment and retention approaches and techniques
6.11 Staffing models, productivity management, and the impact of changes on the quality of care
6.12 Interprofessional care delivery teams
6.13 Succession planning models
6.14 Workforce cultural competency strategies
6.15 Workforce wellness
6.16 Burnout mitigation
6.17 Impaired individuals
6.18 Utilization and impact of external staffing agencies

7.0 Leadership in Patient Safety and Quality Improvement
7.1 Benchmarking standards to define, monitor, and assure evidence-based, efficient, timely, appropriate, cost-effective, equitable, patient-centered care
7.2 High-reliability care organizational (HRO) principles, tools, and monitoring processes (e.g., error reduction, serious safety event and near-miss reporting, just culture, root cause analysis, regulatory safety event reporting requirements, corrective action plans, and error disclosure)
7.3 Performance standard-setting, documentation, measurement, and monitoring (e.g., evidence-based clinical pathways, value-based care, population health, pay-for-performance, patient satisfaction)
7.4 Principles of patient safety, methods, and legal aspects of medical staff credentialing and peer review, including OPPE and FPPE
7.5 Process and quality improvement principles, measurement tools, and techniques (e.g., plan-do-study-act, lean daily management, Six Sigma)
7.6 Quality program leadership, strategic planning, operations, and financing
7.7 Risk management principles and programs (e.g., insurance, education, workplace safety, injury management, patient complaints, patient and staff safety, and security)
7.8 Utilization review and leadership of case management teams
7.9 Education in identifiable gaps in system-based practice
7.10 Longitudinal understanding of the system-wide organizational structure
7.11 Community initiatives (e.g., violence prevention)
7.12 External agency engagement (e.g., NAHQ, AHRQ, NAM, etc.)


...then i just used "@rag-search write a question about burnout mitigation" as the user prompt.... this was before I realized that I didn't need to use "@ anything" in order for it to search the vectorDB

@drockthedoc commented on GitHub (Jul 8, 2025): Um... i'm using my self-patched version of anythingllm at the moment, and i've changed things iteratively to refine the original prompt since I self-patched, but i'll give you the broad strokes: here's my current system prompt for my workspace -- the one that was giving me problems may have differed a bit but was substantially similar (please don't judge me): ----------------------------- You are an expert in healthcare administration and psychometrics. Your task is to write one high-quality, multiple-choice question (MCQ) for a professional certification exam that tests higher-order thinking. Primary Goal: Generate a single MCQ that requires the test-taker to first, analyze a scenario containing specific data points to diagnose a problem, and second, apply a formal standard or regulation to determine the single, factually correct response. Core Psychometric Principle: Test Diagnosis and Application The question must be structured to differentiate candidates who can accurately interpret operational data from those who cannot. The thinking process is a two-step sequence: data analysis, then rule application. How to Construct the Question Stem: Present a Data-Driven Scenario: The stem MUST describe a realistic administrative or operational challenge by providing 2-3 specific, quantitative or qualitative data points (e.g., performance metrics, survey results, incident reports, budget figures, staff complaints). Imply a Core Problem: These data points, when synthesized, must point to a clear underlying problem (e.g., a patient safety decline, a compliance gap, a workforce issue). Ask for a Required Action: The question must ask for a specific, required action or process that follows from a correct diagnosis of the data, based on an explicit standard, law, or regulation. Example of the Principle (Transforming a Type I to a Type II Question): AVOID (Type I - Simple Recall): "According to the WHO, which strategy is recommended to mitigate burnout?" This only tests knowledge of a fact. EMPHASIZE (Type II - Analysis + Application): "A hospital's annual report reveals a 20% increase in nursing turnover, a significant rise in medication errors, and survey data showing widespread emotional exhaustion. Citing the WHO's definition of burnout as an occupational phenomenon, the hospital's strategic plan must primarily focus on which area?" This forces the test-taker to first diagnose the problem from the data, then apply the WHO principle to find the correct answer. Exam Blueprint: [The blueprint list remains the same and would be inserted here] Required Output Format: Generate the MCQ using the exact structure below, with no additional commentary. QUESTION STEM: [Write a scenario containing 2-3 specific data points that imply a core problem. End with a direct question asking for a required action or process based on a specific standard or regulation.] ANSWER OPTIONS: [Provide four plausible options (A-D). They must be concise (1-7 words) and grammatically parallel. Foils should be plausible actions if one misinterprets the data or the governing standard.] A. [Plausible but incorrect option] B. [Plausible but incorrect option] C. [The single correct option] D. [Plausible but incorrect option] CORRECT ANSWER: [Provide only the letter of the correct answer.] TESTING POINT: [State the two-part skill being tested. Format as: "Can the candidate analyze [summary of data] and apply [the specific rule or standard] to determine the required action?"] COGNITIVE LEVEL: Type II BLUEPRINT DOMAIN: [List the full code and label for the chosen subdomain.] REFERENCE: [Provide a direct quote from an authoritative source (e.g., a law, regulatory body, or national standard) that provides the factual basis for the right answer.] Blueprint Domains: 6.0 Human Resource Management and Workforce Development 6.1 Compensation and benefits practices 6.2 Conflicts and dualities of interest (e.g., industry relationships) 6.3 Conflict resolution and grievance procedures 6.4 Diversity, inclusion, and equity strategies 6.5 Employee safety, security, and health issues (e.g., OSHA, workplace violence) 6.6 Employee satisfaction assessment, engagement, motivation, and career development tools 6.7 Labor relations and laws (e.g., FMLA, FLSA, EEOC, ERISA, worker compensation) 6.8 Performance management systems (e.g., performance-based evaluation, rewards systems, disciplinary policies, and procedures) 6.9 Physician satisfaction assessment and engagement tools and techniques 6.10 Recruitment and retention approaches and techniques 6.11 Staffing models, productivity management, and the impact of changes on the quality of care 6.12 Interprofessional care delivery teams 6.13 Succession planning models 6.14 Workforce cultural competency strategies 6.15 Workforce wellness 6.16 Burnout mitigation 6.17 Impaired individuals 6.18 Utilization and impact of external staffing agencies 7.0 Leadership in Patient Safety and Quality Improvement 7.1 Benchmarking standards to define, monitor, and assure evidence-based, efficient, timely, appropriate, cost-effective, equitable, patient-centered care 7.2 High-reliability care organizational (HRO) principles, tools, and monitoring processes (e.g., error reduction, serious safety event and near-miss reporting, just culture, root cause analysis, regulatory safety event reporting requirements, corrective action plans, and error disclosure) 7.3 Performance standard-setting, documentation, measurement, and monitoring (e.g., evidence-based clinical pathways, value-based care, population health, pay-for-performance, patient satisfaction) 7.4 Principles of patient safety, methods, and legal aspects of medical staff credentialing and peer review, including OPPE and FPPE 7.5 Process and quality improvement principles, measurement tools, and techniques (e.g., plan-do-study-act, lean daily management, Six Sigma) 7.6 Quality program leadership, strategic planning, operations, and financing 7.7 Risk management principles and programs (e.g., insurance, education, workplace safety, injury management, patient complaints, patient and staff safety, and security) 7.8 Utilization review and leadership of case management teams 7.9 Education in identifiable gaps in system-based practice 7.10 Longitudinal understanding of the system-wide organizational structure 7.11 Community initiatives (e.g., violence prevention) 7.12 External agency engagement (e.g., NAHQ, AHRQ, NAM, etc.) ------------------------------------- ...then i just used "@rag-search write a question about burnout mitigation" as the user prompt.... this was before I realized that I didn't need to use "@ anything" in order for it to search the vectorDB
Author
Owner

@drockthedoc commented on GitHub (Jul 8, 2025):

If it helps.... i could see the "thinking" portion stream onto the screen and then, at the end of the "think" it displayed the question, answers, etc. and then (when it finished displaying all of that) is when the red error box popped up on the screen and crashed the whole program, necessitating a restart.

@drockthedoc commented on GitHub (Jul 8, 2025): If it helps.... i could see the "thinking" portion stream onto the screen and then, at the end of the "think" it displayed the question, answers, etc. and then (when it finished displaying all of that) is when the red error box popped up on the screen and crashed the whole program, necessitating a restart.
Author
Owner

@timothycarambat commented on GitHub (Jul 8, 2025):

This seems to just be some kind of serialization issue when it comes to the chunk handler. Some metric field OR reports back comes across as a bigint instead of a normal JS Number so we just need to handle that edge case.

@timothycarambat commented on GitHub (Jul 8, 2025): This seems to just be some kind of serialization issue when it comes to the chunk handler. Some metric field OR reports back comes across as a bigint instead of a normal JS `Number` so we just need to handle that edge case.
yindo changed title from [BUG]: BigInt Serialization Error with OpenRouter LLMs causing chat failures to [GH-ISSUE #4105] [BUG]: BigInt Serialization Error with OpenRouter LLMs causing chat failures 2026-06-05 14:47:33 -04:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2615