Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #5760] [MERGED] fix: strip XML-illegal control characters from generated documents #5557

New Issue
Closed
opened 2026-06-05 15:21:44 -04:00 by yindo · 0 comments
yindo commented 2026-06-05 15:21:44 -04:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/5760
Author: @sanidhyasin
Created: 6/5/2026
Status: ✅ Merged
Merged: 6/5/2026
Merged by: @timothycarambat

Base: master ← Head: 5756-strip-invalid-xml-chars-from-generated-files

📝 Commits (4)

  • 8e3bbf2 fix: strip XML-illegal control characters from generated documents
  • 83cb5b1 unset binary format
  • e822766 update test suite to catch drifts
  • ad46030 lint

📊 Changes

5 files changed (+143 additions, -1 deletions)

View changed files

➕ server/__tests__/utils/agents/aibitat/plugins/create-files/lib.test.js (+86 -0)
📝 server/utils/agents/aibitat/plugins/create-files/docx/create-docx-file.js (+7 -0)
📝 server/utils/agents/aibitat/plugins/create-files/lib.js (+33 -0)
📝 server/utils/agents/aibitat/plugins/create-files/pptx/create-presentation.js (+12 -1)
📝 server/utils/agents/aibitat/plugins/create-files/xlsx/create-excel-file.js (+5 -0)

📄 Description

Pull Request Type

  • 🐛 fix (Bug fix)

Relevant Issues

resolves #5756

Description

OOXML documents (.docx/.xlsx/.pptx) embed their text directly into internal XML parts (e.g. word/document.xml). XML 1.0 §2.2 forbids every C0 control character except tab (U+0009), line feed (U+000A), and carriage return (U+000D).

When a user asks the Document Creation agent for content containing LaTeX math such as $\frac{3}{5}$, the model's tool-call arguments arrive as JSON. The \f in \frac is a valid JSON escape that decodes to a form feed (U+000C), so the form feed lands in content before it ever reaches the generator. The resulting file is a structurally valid ZIP, but Office refuses to open it:

Word experienced an error trying to open the file.

This is a silent failure from the user's perspective — the file downloads fine but cannot be opened. It affects any document-generation request with math/LaTeX content (very common for education use cases), and the same illegal-character class (\b, \v, etc.) can reach the other OOXML generators too.

Fix

Add a shared stripInvalidXmlChars helper to the create-files manager (lib.js) that removes XML 1.0–illegal control characters from a string or, recursively, from every string within a nested array/object. It preserves the three legal whitespace control characters (tab, LF, CR) and leaves non-string scalars untouched.

The helper is applied to the user/LLM-supplied content before generation in the handlers that embed text into XML/HTML:

  • create-docx-file.js — content, title, subtitle, author
  • create-pdf-file.js — content (markdown → HTML → PDF)
  • create-excel-file.js — csvData and the sheets array (cell content + sheet names)
  • create-pptx-presentation.js — title, author, and the assembled sub-agent slide data

create-text-file.js is intentionally left untouched, since control characters are legal in plain text/CSV output.

A .docx request for a fractions quiz now opens normally and renders the math as plain text (e.g. rac{3}{5}) instead of producing a corrupt file.

Visuals (if applicable)

N/A

Additional Information

The change is additive and non-breaking: clean content is returned byte-for-byte unchanged. A regression test suite for stripInvalidXmlChars is included at server/__tests__/utils/agents/aibitat/plugins/create-files/lib.test.js.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated (if applicable)
  • I have tested my code functionality
  • Docker build succeeds locally

Focused validation run:

npx --yes jest@29 server/__tests__/utils/agents/aibitat/plugins/create-files/lib.test.js

PASS  __tests__/utils/agents/aibitat/plugins/create-files/lib.test.js
  CreateFilesManager.stripInvalidXmlChars
    ✓ removes the form feed produced by a LaTeX backslash sequence
    ✓ strips every disallowed C0 control character
    ✓ preserves tab, line feed, and carriage return (the legal C0 chars)
    ✓ leaves clean strings unchanged
    ✓ recursively cleans arrays and nested objects
    ✓ returns non-string scalars untouched

Tests:       6 passed, 6 total

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/5760 **Author:** [@sanidhyasin](https://github.com/sanidhyasin) **Created:** 6/5/2026 **Status:** ✅ Merged **Merged:** 6/5/2026 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `5756-strip-invalid-xml-chars-from-generated-files` --- ### 📝 Commits (4) - [`8e3bbf2`](https://github.com/Mintplex-Labs/anything-llm/commit/8e3bbf2790588262019444ecb02ff1514d0487a2) fix: strip XML-illegal control characters from generated documents - [`83cb5b1`](https://github.com/Mintplex-Labs/anything-llm/commit/83cb5b17628354ed8ec782cbd2b69df05e14a668) unset binary format - [`e822766`](https://github.com/Mintplex-Labs/anything-llm/commit/e822766c6d461660174be5cb381c2b7797dad8ac) update test suite to catch drifts - [`ad46030`](https://github.com/Mintplex-Labs/anything-llm/commit/ad46030ec18da32686eae73c900f45f52eafb30d) lint ### 📊 Changes **5 files changed** (+143 additions, -1 deletions) <details> <summary>View changed files</summary> ➕ `server/__tests__/utils/agents/aibitat/plugins/create-files/lib.test.js` (+86 -0) 📝 `server/utils/agents/aibitat/plugins/create-files/docx/create-docx-file.js` (+7 -0) 📝 `server/utils/agents/aibitat/plugins/create-files/lib.js` (+33 -0) 📝 `server/utils/agents/aibitat/plugins/create-files/pptx/create-presentation.js` (+12 -1) 📝 `server/utils/agents/aibitat/plugins/create-files/xlsx/create-excel-file.js` (+5 -0) </details> ### 📄 Description ### Pull Request Type - [x] 🐛 fix (Bug fix) ### Relevant Issues resolves #5756 ### Description OOXML documents (`.docx`/`.xlsx`/`.pptx`) embed their text directly into internal XML parts (e.g. `word/document.xml`). XML 1.0 §2.2 forbids every C0 control character except tab (`U+0009`), line feed (`U+000A`), and carriage return (`U+000D`). When a user asks the Document Creation agent for content containing LaTeX math such as `$\frac{3}{5}$`, the model's tool-call arguments arrive as JSON. The `\f` in `\frac` is a valid JSON escape that decodes to a **form feed (`U+000C`)**, so the form feed lands in `content` before it ever reaches the generator. The resulting file is a structurally valid ZIP, but Office refuses to open it: > Word experienced an error trying to open the file. This is a silent failure from the user's perspective — the file downloads fine but cannot be opened. It affects any document-generation request with math/LaTeX content (very common for education use cases), and the same illegal-character class (`\b`, `\v`, etc.) can reach the other OOXML generators too. ### Fix Add a shared `stripInvalidXmlChars` helper to the create-files manager (`lib.js`) that removes XML 1.0–illegal control characters from a string or, recursively, from every string within a nested array/object. It preserves the three legal whitespace control characters (tab, LF, CR) and leaves non-string scalars untouched. The helper is applied to the user/LLM-supplied content before generation in the handlers that embed text into XML/HTML: - `create-docx-file.js` — `content`, `title`, `subtitle`, `author` - `create-pdf-file.js` — `content` (markdown → HTML → PDF) - `create-excel-file.js` — `csvData` and the `sheets` array (cell content + sheet names) - `create-pptx-presentation.js` — `title`, `author`, and the assembled sub-agent slide data `create-text-file.js` is intentionally left untouched, since control characters are legal in plain text/CSV output. A `.docx` request for a fractions quiz now opens normally and renders the math as plain text (e.g. `rac{3}{5}`) instead of producing a corrupt file. ### Visuals (if applicable) N/A ### Additional Information The change is additive and non-breaking: clean content is returned byte-for-byte unchanged. A regression test suite for `stripInvalidXmlChars` is included at `server/__tests__/utils/agents/aibitat/plugins/create-files/lib.test.js`. ### Developer Validations - [ ] I ran `yarn lint` from the root of the repo & committed changes - [ ] Relevant documentation has been updated (if applicable) - [x] I have tested my code functionality - [ ] Docker build succeeds locally Focused validation run: ```sh npx --yes jest@29 server/__tests__/utils/agents/aibitat/plugins/create-files/lib.test.js ``` ``` PASS __tests__/utils/agents/aibitat/plugins/create-files/lib.test.js CreateFilesManager.stripInvalidXmlChars ✓ removes the form feed produced by a LaTeX backslash sequence ✓ strips every disallowed C0 control character ✓ preserves tab, line feed, and carriage return (the legal C0 chars) ✓ leaves clean strings unchanged ✓ recursively cleans arrays and nested objects ✓ returns non-string scalars untouched Tests: 6 passed, 6 total ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-06-05 15:21:44 -04:00
yindo closed this issue 2026-06-05 15:21:44 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5557
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?