stoatchat/self-hosted
1
0
Fork 0
mirror of https://github.com/stoatchat/self-hosted.git synced 2026-07-21 04:25:24 -04:00
Code Issues 42 Packages Projects Releases Wiki Activity

bug: During upgrade i get some errors #84

New Issue
Open
opened 2026-02-16 12:51:22 -05:00 by yindo · 15 comments
yindo commented 2026-02-16 12:51:22 -05:00
Owner
Copy Link
Copy Source

Originally created by @0ktav1us on GitHub (Dec 24, 2024).

Originally assigned to: @insertish on GitHub.

What happened?

hello, please help me finish updrad#84
run the migration: get error

root@1a7e78fdf5bb:/cwd# node ./20240929-autumn-rewrite.mjs 
1 7505 {
  _id: 'o4U--LPqcFkm6NpvCufKn1cmSZbMTUxwQX_qZ7gwDo',
  tag: 'attachments',
  filename: '2023-03-28 12-26-05.mkv',
  metadata: { type: 'File' },
  content_type: 'application/x-matroska',
  size: 1760809,
  message_id: '01GWKW2YNZ1M9DQP66414MH8S6'
}
file:///cwd/20240929-autumn-rewrite.mjs:41
    if (!objectLookup[f.message_id]) {
    ^

ReferenceError: objectLookup is not defined
    at determineUploaderIdAndUse (file:///cwd/20240929-autumn-rewrite.mjs:41:5)
    at file:///cwd/20240929-autumn-rewrite.mjs:347:16
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v21.7.3

And if I continue and run containers, I don't see attachments, it says that the files don't exist test

Originally created by @0ktav1us on GitHub (Dec 24, 2024). Originally assigned to: @insertish on GitHub. ### What happened? hello, please help me finish updrad#84 run the migration: get error ``` root@1a7e78fdf5bb:/cwd# node ./20240929-autumn-rewrite.mjs 1 7505 { _id: 'o4U--LPqcFkm6NpvCufKn1cmSZbMTUxwQX_qZ7gwDo', tag: 'attachments', filename: '2023-03-28 12-26-05.mkv', metadata: { type: 'File' }, content_type: 'application/x-matroska', size: 1760809, message_id: '01GWKW2YNZ1M9DQP66414MH8S6' } file:///cwd/20240929-autumn-rewrite.mjs:41 if (!objectLookup[f.message_id]) { ^ ReferenceError: objectLookup is not defined at determineUploaderIdAndUse (file:///cwd/20240929-autumn-rewrite.mjs:41:5) at file:///cwd/20240929-autumn-rewrite.mjs:347:16 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) Node.js v21.7.3 ``` And if I continue and run containers, I don't see attachments, it says that the files don't exist <a href="https://ibb.co/hXbCh6N"><img src="https://i.ibb.co/Xy1VNQP/test.png" alt="test" border="0"></a>
yindo added the bug label 2026-02-16 12:51:22 -05:00
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@0ktav1us commented on GitHub (Dec 24, 2024):

new images attach and display

@0ktav1us commented on GitHub (Dec 24, 2024): new images attach and display
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@ThomasBaruzier commented on GitHub (Jan 17, 2025):

Same behavior, here's what allowed me to finish the migration:

diff --git a/migrations/20240929-autumn-rewrite.mjs b/migrations/20240929-autumn-rewrite.mjs
index c8643bc..1ab854a 100644
--- a/migrations/20240929-autumn-rewrite.mjs
+++ b/migrations/20240929-autumn-rewrite.mjs
@@ -26,12 +26,38 @@ const BUCKET_MAP = {
  *
  * TODO: change if necessary
  */
-const CONNECTION_URL = "mongodb://database";
+const CONNECTION_URL = "mongodb://database:27017";
 
 const mongo = new MongoClient(CONNECTION_URL);
 await mongo.connect();
 
+const objectLookup = {};
+
+/**
+ * Decode timestamp from Ulid
+ * @param {string} id Ulid string
+ * @returns {number} Timestamp in milliseconds
+ */
+function decodeTime(id) {
+  // First 10 characters of ULID represent milliseconds precision timestamp
+  const timestamp = id.substring(0, 10);
+  // Convert from base32 to decimal
+  const base32Chars = '0123456789ABCDEFGHJKMNPQRSTVWXYZ';
+  let decoded = 0;
+  for (let i = 0; i < timestamp.length; i++) {
+    const char = timestamp[i];
+    const value = base32Chars.indexOf(char.toUpperCase());
+    if (value === -1) {
+      throw new Error(`Invalid ULID character: ${char}`);
+    }
+    decoded = decoded * 32 + value;
+  }
+  // ULID timestamps are in milliseconds
+  return decoded;
+}
+
 async function determineUploaderIdAndUse(f, v, i) {
+  console.log(`Processing file ${i}:`, f._id);
   if (f.tag === "attachments" && v === "attachments") {
     if (typeof f.message_id !== "string") {
       console.warn(i, "No message id specified.");
@@ -39,6 +65,7 @@ async function determineUploaderIdAndUse(f, v, i) {
     }
 
     if (!objectLookup[f.message_id]) {
+      console.log(`Querying database for message ${f.message_id}`);
       objectLookup[f.message_id] = await mongo
         .db("revolt")
         .collection("messages")
@@ -52,6 +79,7 @@ async function determineUploaderIdAndUse(f, v, i) {
       return null;
     }
 
+    console.log(`Found message ${f.message_id} for file ${f._id}`);
     return {
       uploaded_at: new Date(decodeTime(f.message_id)),
       uploader_id: objectLookup[f.message_id].author,
@@ -368,6 +396,7 @@ const files_pt2 = await mongo
   })
   .toArray();
 
+if (files_pt2.length > 0) {
 await mongo
   .db("revolt")
   .collection("attachment_hashes")
@@ -403,3 +432,10 @@ for (const file of files_pt2) {
       }
     );
 }
+} else {
+  console.log("No files to process in files_pt2.");
+}
+
+await mongo.close();
+console.log("Migration completed successfully.");
+process.exit(0);

And while the autumn server base URL responds with {"autumn":"Hello, I am a file server!","version":"0.8.1"}, my assets are still not found:

Image

@ThomasBaruzier commented on GitHub (Jan 17, 2025): Same behavior, here's what allowed me to finish the migration: ```diff diff --git a/migrations/20240929-autumn-rewrite.mjs b/migrations/20240929-autumn-rewrite.mjs index c8643bc..1ab854a 100644 --- a/migrations/20240929-autumn-rewrite.mjs +++ b/migrations/20240929-autumn-rewrite.mjs @@ -26,12 +26,38 @@ const BUCKET_MAP = { * * TODO: change if necessary */ -const CONNECTION_URL = "mongodb://database"; +const CONNECTION_URL = "mongodb://database:27017"; const mongo = new MongoClient(CONNECTION_URL); await mongo.connect(); +const objectLookup = {}; + +/** + * Decode timestamp from Ulid + * @param {string} id Ulid string + * @returns {number} Timestamp in milliseconds + */ +function decodeTime(id) { + // First 10 characters of ULID represent milliseconds precision timestamp + const timestamp = id.substring(0, 10); + // Convert from base32 to decimal + const base32Chars = '0123456789ABCDEFGHJKMNPQRSTVWXYZ'; + let decoded = 0; + for (let i = 0; i < timestamp.length; i++) { + const char = timestamp[i]; + const value = base32Chars.indexOf(char.toUpperCase()); + if (value === -1) { + throw new Error(`Invalid ULID character: ${char}`); + } + decoded = decoded * 32 + value; + } + // ULID timestamps are in milliseconds + return decoded; +} + async function determineUploaderIdAndUse(f, v, i) { + console.log(`Processing file ${i}:`, f._id); if (f.tag === "attachments" && v === "attachments") { if (typeof f.message_id !== "string") { console.warn(i, "No message id specified."); @@ -39,6 +65,7 @@ async function determineUploaderIdAndUse(f, v, i) { } if (!objectLookup[f.message_id]) { + console.log(`Querying database for message ${f.message_id}`); objectLookup[f.message_id] = await mongo .db("revolt") .collection("messages") @@ -52,6 +79,7 @@ async function determineUploaderIdAndUse(f, v, i) { return null; } + console.log(`Found message ${f.message_id} for file ${f._id}`); return { uploaded_at: new Date(decodeTime(f.message_id)), uploader_id: objectLookup[f.message_id].author, @@ -368,6 +396,7 @@ const files_pt2 = await mongo }) .toArray(); +if (files_pt2.length > 0) { await mongo .db("revolt") .collection("attachment_hashes") @@ -403,3 +432,10 @@ for (const file of files_pt2) { } ); } +} else { + console.log("No files to process in files_pt2."); +} + +await mongo.close(); +console.log("Migration completed successfully."); +process.exit(0); ``` And while the autumn server base URL responds with `{"autumn":"Hello, I am a file server!","version":"0.8.1"}`, my assets are still not found: ![Image](https://github.com/user-attachments/assets/468ad6e0-52bc-45bf-b58c-fa29febb4b53)
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@0ktav1us commented on GitHub (Jan 20, 2025):

Image
It worked, thanks, but I get the same error
{"type":"NotFound","location":"crates/services/autumn/src/api.rs:372:20"}

@0ktav1us commented on GitHub (Jan 20, 2025): ![Image](https://github.com/user-attachments/assets/ecd8e2bc-684b-4094-9b7d-5a152d3db7c9) It worked, thanks, but I get the same error `{"type":"NotFound","location":"crates/services/autumn/src/api.rs:372:20"}`
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@ThomasBaruzier commented on GitHub (Jan 20, 2025):

It worked, thanks, but I get the same error {"type":"NotFound","location":"crates/services/autumn/src/api.rs:372:20"}

Well, we're in this together. I'll see what I can do soon enough but i'm pretty sure it's out of my reach.

@ThomasBaruzier commented on GitHub (Jan 20, 2025): > It worked, thanks, but I get the same error `{"type":"NotFound","location":"crates/services/autumn/src/api.rs:372:20"}` Well, we're in this together. I'll see what I can do soon enough but i'm pretty sure it's out of my reach.
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@0ktav1us commented on GitHub (Jan 20, 2025):

Image
// Ignore files that haven't been attached,
According to this condition we get an error

@0ktav1us commented on GitHub (Jan 20, 2025): ![Image](https://github.com/user-attachments/assets/3c9ae247-2c38-4e0f-a707-806dbaaaf07b) // Ignore files that haven't been attached, **According to this condition we get an error**
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@0ktav1us commented on GitHub (Jan 20, 2025):

I think we need to compare this piece of code with the old autumn, see how the attachments worked in the old one

@0ktav1us commented on GitHub (Jan 20, 2025): I think we need to compare this piece of code with the old autumn, see how the attachments worked in the old one
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@ThomasBaruzier commented on GitHub (Jan 20, 2025):

In my opinion, the migration is not doing its job properly because of my manual dirty fix, hence the missing of attachment retrieval and this error

@ThomasBaruzier commented on GitHub (Jan 20, 2025): In my opinion, the migration is not doing its job properly because of my manual dirty fix, hence the missing of attachment retrieval and this error
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@ThomasBaruzier commented on GitHub (Feb 12, 2025):

Hello,

I don't want to put any pressure on this, but it would be great if that issue got solved somehow. I'd be happy to cooperate.

Thank you!

@ThomasBaruzier commented on GitHub (Feb 12, 2025): Hello, I don't want to put any pressure on this, but it would be great if that issue got solved somehow. I'd be happy to cooperate. Thank you!
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@0ktav1us commented on GitHub (Feb 18, 2025):

Please help us to resolve it

@0ktav1us commented on GitHub (Feb 18, 2025): Please help us to resolve it
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@poggingfish commented on GitHub (Mar 6, 2025):

I am having the same issue.. Is there any temporary solution?

@poggingfish commented on GitHub (Mar 6, 2025): I am having the same issue.. Is there any temporary solution?
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@ThomasBaruzier commented on GitHub (Mar 6, 2025):

Not for now, but I haven't tried hard fixing it.
I only fixed the migrarion errors

@ThomasBaruzier commented on GitHub (Mar 6, 2025): Not for now, but I haven't tried hard fixing it. I only fixed the migrarion errors
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@poggingfish commented on GitHub (Apr 28, 2025):

I would really love to see this issue get any amount of attention from the devs, I moved to matrix because I didn't want to have to deal with this.
The devs really need to start focusing more on the self hosting process.

@poggingfish commented on GitHub (Apr 28, 2025): I would really love to see this issue get any amount of attention from the devs, I moved to matrix because I didn't want to have to deal with this. The devs really need to start focusing more on the self hosting process.
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@insertish commented on GitHub (May 12, 2025):

Hi, sorry, I'm quite behind on issues across the entire project.
I've added the missing const to the script but I would need more details to understand the issue at hand.

If someone has a file that appears 'NotFound' and can send the database document here that would be appreciated, I can guide you through exporting the document if needed.

@insertish commented on GitHub (May 12, 2025): Hi, sorry, I'm quite behind on issues across the entire project. I've added the missing const to the script but I would need more details to understand the issue at hand. If someone has a file that appears 'NotFound' and can send the database document here that would be appreciated, I can guide you through exporting the document if needed.
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@ThomasBaruzier commented on GitHub (Jun 2, 2025):

Hi @insertish,

Following up on providing database documents for a "NotFound" attachment

  1. Attachment document (from db.attachments):
    file id: 9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ
{
  "_id": "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ",
  "tag": "attachments",
  "filename": "Screenshot_2024-09-07-19-34-13-389_chat.revolt.jpg",
  "metadata": { "type": "Image", "width": Long("1080"), "height": Long("2340") },
  "content_type": "image/jpeg",
  "size": Long("168327"),
  "message_id": "01J76RT5J8SE9DPGGB74V0EZMN"
}
  1. Message document (from db.messages):
    message id: 01J76RT5J8SE9DPGGB74V0EZMN
{
  "_id": "01J76RT5J8SE9DPGGB74V0EZMN",
  "nonce": "01J76RT5F48TYD01FCD07Z0P03",
  "channel": "01HZYD0FQGKR2S4N5NR8A492K2",
  "author": "01HZXANW0N6WBTSS7NF7EBG2GE",
  "content": "",
  "attachments": [
    {
      "_id": "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ",
      "tag": "attachments",
      "filename": "Screenshot_2024-09-07-19-34-13-389_chat.revolt.jpg",
      "metadata": { "type": "Image", "width": 1080, "height": 2340 },
      "content_type": "image/jpeg",
      "size": 168327
    }
  ]
}
  1. attachment_hashes collection check:
    querying for this attachment in attachment_hashes yields no results:
revolt> db.attachment_hashes.findOne({ file_id: "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ" })
null
revolt> db.attachment_hashes.findOne({ _id: "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ" })
null

Using db.attachment_hashes.find() shows that its first entry is from March 2, 2025, so 3 months ago. There are no older elements to be found.

@ThomasBaruzier commented on GitHub (Jun 2, 2025): Hi @insertish, Following up on providing database documents for a "NotFound" attachment 1. Attachment document (from `db.attachments`): file id: `9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ` ```sql { "_id": "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ", "tag": "attachments", "filename": "Screenshot_2024-09-07-19-34-13-389_chat.revolt.jpg", "metadata": { "type": "Image", "width": Long("1080"), "height": Long("2340") }, "content_type": "image/jpeg", "size": Long("168327"), "message_id": "01J76RT5J8SE9DPGGB74V0EZMN" } ``` 2. Message document (from `db.messages`): message id: `01J76RT5J8SE9DPGGB74V0EZMN` ```sql { "_id": "01J76RT5J8SE9DPGGB74V0EZMN", "nonce": "01J76RT5F48TYD01FCD07Z0P03", "channel": "01HZYD0FQGKR2S4N5NR8A492K2", "author": "01HZXANW0N6WBTSS7NF7EBG2GE", "content": "", "attachments": [ { "_id": "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ", "tag": "attachments", "filename": "Screenshot_2024-09-07-19-34-13-389_chat.revolt.jpg", "metadata": { "type": "Image", "width": 1080, "height": 2340 }, "content_type": "image/jpeg", "size": 168327 } ] } ``` 3. `attachment_hashes` collection check: querying for this attachment in `attachment_hashes` yields no results: ``` revolt> db.attachment_hashes.findOne({ file_id: "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ" }) null revolt> db.attachment_hashes.findOne({ _id: "9MWzftYHh2x8VhcZ0ubm7rJunNeVkz8SOPRn3p0ZqJ" }) null ``` Using `db.attachment_hashes.find()` shows that its first entry is from March 2, 2025, so 3 months ago. There are no older elements to be found.
yindo commented 2026-02-16 12:51:23 -05:00
Author
Owner
Copy Link
Copy Source

@poggingfish commented on GitHub (Aug 27, 2025):

Any plans on fixing this?

@poggingfish commented on GitHub (Aug 27, 2025): Any plans on fixing this?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
good first issue
help wanted
improvement
pull-request
Mirrored from GitHub Pull Request
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stoatchat/self-hosted#84
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?