Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1627] [BUG]: API json output messing up escaped CR and LF #1057

New Issue
Closed
opened 2026-02-22 18:22:55 -05:00 by yindo · 7 comments
yindo commented 2026-02-22 18:22:55 -05:00
Owner
Copy Link
Copy Source

Originally created by @cyberlink1 on GitHub (Jun 6, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1627

How are you running AnythingLLM?

Docker (remote machine)

What happened?

Im working on getting anything-llm connected to n8n to process email requests.
The test workspace prompt includes the default plus "The body should be in a business email format . Output must be in JSON"

When I do the api call "/api/v1/workspace/xxxxxx/chat" to send it a message and get the response, the escaped CR and LF in the response are wrong. Sometimes it misses the escape before n or r sometimes it leaves off the n or r and sometimes it includes an \m.

Here is an example return I just pulled.

[
  {
    "id": "326e9409-ef25-44dc-9216-7fa8e31624ee",
    "type": "textResponse",
    "close": true,
    "error": null,
    "chatId": 111,
    "textResponse": "{  \"subject\":  \"Re: XXXXXX Services\",  \"body\":  \"Dear XXXX,\\r\\nThe phone number I provided earlier, (555) 123-4567, is indeed a valid phone number for our Client Engagement Team. If you'd like to schedule a call or have any further questions, please don't hesitate to reach out.\\r\\n\\r\\The email address, xxxxxxxxxxxx@xxxxxxxxxxxx.com, remains the best contact method for signing up for our services.\\r\\n\\r\\If there's anything else I can assist you with, please feel free to ask.\\r\\n\\r\\Best regards,\\r\\nxxxxxx\" }",
    "sources": []
  }
]

Just before "Best regards" you can see what I am talking about as there is an escape before the B in best, there should be an n there. There appear to be 3 such instances in the textResponse and it fails the json check.

Are there known steps to reproduce?

No response

Originally created by @cyberlink1 on GitHub (Jun 6, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1627 ### How are you running AnythingLLM? Docker (remote machine) ### What happened? Im working on getting anything-llm connected to n8n to process email requests. The test workspace prompt includes the default plus "The body should be in a business email format . Output must be in JSON" When I do the api call "/api/v1/workspace/xxxxxx/chat" to send it a message and get the response, the escaped CR and LF in the response are wrong. Sometimes it misses the escape before n or r sometimes it leaves off the n or r and sometimes it includes an \m. Here is an example return I just pulled. ``` [ { "id": "326e9409-ef25-44dc-9216-7fa8e31624ee", "type": "textResponse", "close": true, "error": null, "chatId": 111, "textResponse": "{ \"subject\": \"Re: XXXXXX Services\", \"body\": \"Dear XXXX,\\r\\nThe phone number I provided earlier, (555) 123-4567, is indeed a valid phone number for our Client Engagement Team. If you'd like to schedule a call or have any further questions, please don't hesitate to reach out.\\r\\n\\r\\The email address, xxxxxxxxxxxx@xxxxxxxxxxxx.com, remains the best contact method for signing up for our services.\\r\\n\\r\\If there's anything else I can assist you with, please feel free to ask.\\r\\n\\r\\Best regards,\\r\\nxxxxxx\" }", "sources": [] } ] ``` Just before "Best regards" you can see what I am talking about as there is an escape before the B in best, there should be an n there. There appear to be 3 such instances in the textResponse and it fails the json check. ### Are there known steps to reproduce? _No response_
yindo added the possible buginvestigating labels 2026-02-22 18:22:55 -05:00
yindo closed this issue 2026-02-22 18:22:56 -05:00
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 7, 2024):

This is because the file uploaded has the raw-text of \n \r \t in it and when being JSON stringified they are being properly escaped. What kind of file are you uploading that has this content in it?

@timothycarambat commented on GitHub (Jun 7, 2024): This is because the file uploaded has the raw-text of `\n \r \t` in it and when being JSON `stringified` they are being properly escaped. What kind of file are you uploading that has this content in it?
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@cyberlink1 commented on GitHub (Jun 7, 2024):

This is because the file uploaded has the raw-text of \n \r \t in it and when being JSON stringified they are being properly escaped. What kind of file are you uploading that has this content in it?

I did not upload a file, I scraped a website for the information.
I find it had to believe that "\\Best Regards" is a proper escape.

@cyberlink1 commented on GitHub (Jun 7, 2024): > This is because the file uploaded has the raw-text of `\n \r \t` in it and when being JSON `stringified` they are being properly escaped. What kind of file are you uploading that has this content in it? I did not upload a file, I scraped a website for the information. I find it had to believe that "\\\Best Regards" is a proper escape.
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 7, 2024):

I find it had to believe that "\Best Regards" is a proper escape.

Its clearly not, but what I am saying is that the content itself has a literal \ in it, so it's being escaped. What is the link?

The raw content is coming across as \Best Regards - why there is a slash, I'm not sure. All we do is JSON.Stringify so we do no additional parsing on top of that

@timothycarambat commented on GitHub (Jun 7, 2024): > I find it had to believe that "\\Best Regards" is a proper escape. Its clearly not, but what I am saying is that the _content_ itself has a literal `\` in it, so it's being escaped. What is the link? The raw content is coming across as `\Best Regards` - why there is a slash, I'm not sure. All we do is `JSON.Stringify` so we do no additional parsing on top of that
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@cyberlink1 commented on GitHub (Jun 7, 2024):

The raw content is coming across as \Best Regards - why there is a slash, I'm not sure. All we do is JSON.Stringify so we do no additional parsing on top of that

That is an AI generated email salutation. It is not part of anything scraped. I scraped a company called vizientinc to load the data for testing. Something to do with healthcare and was a decent size. I scraped 3 deep and 20 links.

@cyberlink1 commented on GitHub (Jun 7, 2024): > The raw content is coming across as `\Best Regards` - why there is a slash, I'm not sure. All we do is `JSON.Stringify` so we do no additional parsing on top of that That is an AI generated email salutation. It is not part of anything scraped. I scraped a company called vizientinc to load the data for testing. Something to do with healthcare and was a decent size. I scraped 3 deep and 20 links.
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 7, 2024):

@cyberlink1 I apologize, this entire time I thought we had been talking about content scraped from a website not a response. I just got off a long flight back from China so I'm still jet lagged.

Okay, so the response from the LLM somehow resulted in a \\Best Regards in the response. Which is indeed weird. What model are you using for chatting?

So with textResponse we simply save the output - we don't apply any type of processing on it. LLMs are surprisingly bad at generating reliable JSON output. Its a persistent problem and I simply think all that occurred here is a hallucination. That is it!

Here is what we do to try to alleviate bad JSON hallucinations since LLMs, especially OSS LLMs, are horrendous at just making decent JSON and most provides don't have a JSON mode that just works.

https://github.com/Mintplex-Labs/anything-llm/blob/3434bf4686901721b109b7d67b771ef2957f264c/server/utils/http/index.js#L66

Feel free to borrow that function. In fact, when i use it on your output it gets a better result:

{
  subject: "Re: XXXXXX Services",
  body: "Dear XXXX,\r\n
    The phone number I provided earlier, (555) 123-4567, is indeed a valid phone number for our Client Engagement Team. If you'd like to schedule a call or have any further questions, please don't hesitate to reach out.\r\n
    \rThe email address, xxxxxxxxxxxx@xxxxxxxxxxxx.com, remains the best contact method for signing up for our services.\r\n
    \rIf there's anything else I can assist you with, please feel free to ask.\r\n
    \rBest regards,\r\n
    xxxxxx"
}
@timothycarambat commented on GitHub (Jun 7, 2024): @cyberlink1 I apologize, this entire time I thought we had been talking about content scraped from a _website_ not a response. I just got off a long flight back from China so I'm still jet lagged. Okay, so the **response** from the LLM somehow resulted in a `\\Best Regards` in the response. Which is indeed weird. What model are you using for chatting? So with `textResponse` we simply save the _output_ - we don't apply any type of processing on it. LLMs are surprisingly bad at generating reliable JSON output. Its a persistent problem and I simply think all that occurred here is a hallucination. That is it! Here is what we do to try to alleviate bad JSON hallucinations since LLMs, **especially OSS LLMs**, are horrendous at just making decent JSON and most provides don't have a JSON mode that just works. https://github.com/Mintplex-Labs/anything-llm/blob/3434bf4686901721b109b7d67b771ef2957f264c/server/utils/http/index.js#L66 Feel free to borrow that function. In fact, when i use it on your output it gets a better result: ``` { subject: "Re: XXXXXX Services", body: "Dear XXXX,\r\n The phone number I provided earlier, (555) 123-4567, is indeed a valid phone number for our Client Engagement Team. If you'd like to schedule a call or have any further questions, please don't hesitate to reach out.\r\n \rThe email address, xxxxxxxxxxxx@xxxxxxxxxxxx.com, remains the best contact method for signing up for our services.\r\n \rIf there's anything else I can assist you with, please feel free to ask.\r\n \rBest regards,\r\n xxxxxx" } ```
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@cyberlink1 commented on GitHub (Jun 7, 2024):

No worries, I know how jetlag can be.

Im using ollama with minstrel as the llm. I tested the backend ollama and the escapes were not in the response, I then tested from anythingllm and found the escapes. Trying to narrow down where they were being injected.

Ill see what I can do with the function, Im not well versed in java.

@cyberlink1 commented on GitHub (Jun 7, 2024): No worries, I know how jetlag can be. Im using ollama with minstrel as the llm. I tested the backend ollama and the escapes were not in the response, I then tested from anythingllm and found the escapes. Trying to narrow down where they were being injected. Ill see what I can do with the function, Im not well versed in java.
yindo commented 2026-02-22 18:22:56 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 7, 2024):

They for sure came from the LLM result directly. Since LLMs are non-deterministic you'll probably get a slightly different result every time even when using Ollama directly or via AnythingLLM. Makes solving the problem even harder since you might get good JSON directly 8/10 times, but the other 2 times are close but not strictly valid so they need to be attempted to be repaired and if that fails - assume the response is text.

In your case, since you just want JSON and nothing else you can attempt to parse/repair the response and if its not JSON - exit the function early.

@timothycarambat commented on GitHub (Jun 7, 2024): They for sure came from the LLM result directly. Since LLMs are non-deterministic you'll probably get a slightly different result every time even when using Ollama directly or via AnythingLLM. Makes solving the problem even harder since you might get good JSON directly 8/10 times, but the other 2 times are _close_ but not strictly valid so they need to be attempted to be repaired and if that fails - assume the response is text. In your case, since you just want JSON and nothing else you can attempt to parse/repair the response and if its not JSON - exit the function early.
yindo changed title from [BUG]: API json output messing up escaped CR and LF to [GH-ISSUE #1627] [BUG]: API json output messing up escaped CR and LF 2026-06-05 14:38:45 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1057
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?