Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #183] [CLOSED] Feature/metadata #3234

New Issue
Closed
opened 2026-02-22 18:33:20 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:33:20 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/183
Author: @franzbischoff
Created: 8/10/2023
Status: ❌ Closed

Base: master ← Head: feature/metadata

📝 Commits (10+)

  • 61cbae5 cosmetic changes to be compatible to hadolint
  • 0bd1cfe common configuration for most editors until better plugins comes up
  • f767989 Changes on PDF metadata, using PyMuPDF (faster and more compatible)
  • b7d268c small changes on other file ingestions in order to try to keep the fields equal
  • 292bb56 Lint, review, and review
  • bb7f4c0 Merge branch 'Mintplex-Labs:master' into feature/metadata
  • fbdd35d Merge branch 'Mintplex-Labs:master' into feature/metadata
  • d7af56b Merge branch 'Mintplex-Labs:master' into feature/metadata
  • b3e21e1 Merge branch 'Mintplex-Labs:master' into feature/metadata
  • 09e499a Merge branch 'Mintplex-Labs:master' into feature/metadata

📊 Changes

11 files changed (+139 additions, -59 deletions)

View changed files

➕ .editorconfig (+12 -0)
📝 clean.sh (+0 -0)
📝 collector/requirements.txt (+4 -2)
📝 collector/scripts/gitbook.py (+4 -4)
📝 collector/scripts/watch/convert/as_docx.py (+13 -7)
📝 collector/scripts/watch/convert/as_markdown.py (+8 -5)
📝 collector/scripts/watch/convert/as_mbox.py (+7 -5)
📝 collector/scripts/watch/convert/as_pdf.py (+36 -8)
📝 collector/scripts/watch/convert/as_text.py (+21 -5)
📝 docker/Dockerfile (+33 -22)
📝 package.json (+1 -1)

📄 Description

As discussed with @timothycarambat , this implements the PyMuPDF that is quick and handles Unicode better.
Also, allows us to use actual PDF metadata as 'title', 'author', etc, to populate the embedded metadata for further references when searching.

  • One thing I find out is that the window that shows the references of the searches is not handling Unicode on 'title' field, but it is on the text body. This should be fixed.

  • Known bug: lanceDB insists in assume the Scheme is different when differerent filetypes are uploaded for embedding. Pinecone handle this ok.

I've reviewed the documentation and the actual tables created by lanceDB and could not figure out.

The rest works fine.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/183 **Author:** [@franzbischoff](https://github.com/franzbischoff) **Created:** 8/10/2023 **Status:** ❌ Closed **Base:** `master` ← **Head:** `feature/metadata` --- ### 📝 Commits (10+) - [`61cbae5`](https://github.com/Mintplex-Labs/anything-llm/commit/61cbae52b079568eb4848debebcdd270a4cf5514) cosmetic changes to be compatible to hadolint - [`0bd1cfe`](https://github.com/Mintplex-Labs/anything-llm/commit/0bd1cfe8b306469f6764b26202191b2cfab8a8bc) common configuration for most editors until better plugins comes up - [`f767989`](https://github.com/Mintplex-Labs/anything-llm/commit/f767989eb75688b4990475a6233db3e985560f19) Changes on PDF metadata, using PyMuPDF (faster and more compatible) - [`b7d268c`](https://github.com/Mintplex-Labs/anything-llm/commit/b7d268cbd2908372ffe529044e7ae1e8bfa4f64d) small changes on other file ingestions in order to try to keep the fields equal - [`292bb56`](https://github.com/Mintplex-Labs/anything-llm/commit/292bb563bf09f07cee78e0e93b9b5560b004c8f5) Lint, review, and review - [`bb7f4c0`](https://github.com/Mintplex-Labs/anything-llm/commit/bb7f4c0955b5e2239189c19e80a23cd3da27870c) Merge branch 'Mintplex-Labs:master' into feature/metadata - [`fbdd35d`](https://github.com/Mintplex-Labs/anything-llm/commit/fbdd35d3c87d6dc6772e15445fd74a3cad1570b8) Merge branch 'Mintplex-Labs:master' into feature/metadata - [`d7af56b`](https://github.com/Mintplex-Labs/anything-llm/commit/d7af56bcb5f3913401e5bc54e91c2d6d90577dc8) Merge branch 'Mintplex-Labs:master' into feature/metadata - [`b3e21e1`](https://github.com/Mintplex-Labs/anything-llm/commit/b3e21e12f9324886302f87db83d3e603b8c4c4c2) Merge branch 'Mintplex-Labs:master' into feature/metadata - [`09e499a`](https://github.com/Mintplex-Labs/anything-llm/commit/09e499aa4e460b017879f959e773c300bd42d126) Merge branch 'Mintplex-Labs:master' into feature/metadata ### 📊 Changes **11 files changed** (+139 additions, -59 deletions) <details> <summary>View changed files</summary> ➕ `.editorconfig` (+12 -0) 📝 `clean.sh` (+0 -0) 📝 `collector/requirements.txt` (+4 -2) 📝 `collector/scripts/gitbook.py` (+4 -4) 📝 `collector/scripts/watch/convert/as_docx.py` (+13 -7) 📝 `collector/scripts/watch/convert/as_markdown.py` (+8 -5) 📝 `collector/scripts/watch/convert/as_mbox.py` (+7 -5) 📝 `collector/scripts/watch/convert/as_pdf.py` (+36 -8) 📝 `collector/scripts/watch/convert/as_text.py` (+21 -5) 📝 `docker/Dockerfile` (+33 -22) 📝 `package.json` (+1 -1) </details> ### 📄 Description As discussed with @timothycarambat , this implements the PyMuPDF that is quick and handles Unicode better. Also, allows us to use actual PDF metadata as 'title', 'author', etc, to populate the embedded metadata for further references when searching. - One thing I find out is that the window that shows the references of the searches is not handling Unicode on 'title' field, but it is on the text body. This should be fixed. - Known bug: lanceDB insists in assume the Scheme is different when differerent filetypes are uploaded for embedding. Pinecone handle this ok. I've reviewed the documentation and the actual tables created by lanceDB and could not figure out. The rest works fine. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-22 18:33:20 -05:00
yindo closed this issue 2026-02-22 18:33:20 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3234
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?