Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2791] [FEAT]: Session Management with AnythingLLM and API #1788

New Issue
Closed
opened 2026-02-22 18:26:32 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:26:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @Peterson047 on GitHub (Dec 10, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2791

What would you like to see?

Hello everyone!

I've been working on a project for months and I've encountered a challenge with session management. I tested a solution using Dialogflow CX and was able to use the JSON from the request to define the session via WhatsApp, where the person's number becomes the thread ID. However, when I tried to implement something similar in AnythingLLM, I couldn't find a ready-made solution for this.

To get around the problem, I used the thread creation and message sending APIs. My webhook is hosted on Google Cloud Functions and receives requests via Make (Integromat). Since there is no API to list existing threads, the code handles errors assuming that the thread already exists and, thus, concatenates a link to send the request.

My question is: is there a better way to manage sessions in a more optimized way? My solution works well, but the lack of an API to list the threads created in a workspace makes the process difficult. Is the team already working on something along these lines, or is there an alternative approach I can apply to improve session management?

I welcome any suggestions!

My code:

from http.server import BaseHTTPRequestHandler, HTTPServer
import requests

# AnythingLLM API configurations
API_BASE_URL = "http://YOUR_DOMAIN/api/v1/workspace"
WORKSPACE_SLUG = "WKP-NAME"
API_KEY = "API-ANYTHING"

# Simulates an in-memory database to associate wa_id with threads
user_threads = {}

def create_thread(name, slug):
    """Creates a new thread in AnythingLLM."""
    url = f"{API_BASE_URL}/{WORKSPACE_SLUG}/thread/new"
    headers = {
        "accept": "application/json",
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    data = {
        "userId": 1,  # This is fixed, can be changed depending on the case
        "name": name,
        "slug": slug
    }
    response = requests.post(url, headers=headers, json=data)
    if response.status_code == 200:
        return response.json()["thread"]
    else:
        raise Exception(f"Error creating thread: {response.text}")

def send_message_to_thread(workspace_slug, thread_slug, message):
    """Sends a message to a thread in AnythingLLM."""
    url = f"{API_BASE_URL}/{workspace_slug}/thread/{thread_slug}/chat"
    headers = {
        "accept": "application/json",
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }
    data = {
        "message": message,
        "mode": "chat",  # Can be 'chat' or 'query'
        "userId": 1  # This is fixed, can be changed depending on the case
    }
    response = requests.post(url, headers=headers, json=data)
    if response.status_code == 200:
        return response.json()
    else:
        raise Exception(f"Error sending message: {response.text}")

class WebhookHandler(BaseHTTPRequestHandler):
    def do_POST(self):
        """Handles incoming POST requests to the webhook."""
        content_length = int(self.headers["Content-Length"])
        post_data = self.rfile.read(content_length)
        try:
            request_body = json.loads(post_data)
            wa_id = request_body[0]["contacts"][0]["wa_id"]
            name = request_body[0]["contacts"][0]["profile"]["name"]
            message_text = request_body[0]["messages"][0]["text"]["body"]

            # Checks if the user already has a thread
            if wa_id not in user_threads:
                try:
                    # Creates a new thread for the user
                    thread = create_thread(name, wa_id)
                    user_threads[wa_id] = thread["slug"]
                    print(f"New thread created for {wa_id}: {thread['slug']}")
                except Exception as e:
                    print(f"Error creating thread for {wa_id}: {e}")
                    if 'NoneType' in str(e):  # If the error is related to 'NoneType'
                        # Assumes the thread already exists and associates it
                        print(f"The thread already exists for {wa_id}, associating with slug")
                        user_threads[wa_id] = wa_id  # Uses wa_id as slug to associate with the existing thread
            else:
                print(f"Existing thread for {wa_id}: {user_threads[wa_id]}")

            # Retrieves the associated thread
            thread_slug = user_threads[wa_id]

            # Sends the message to the associated thread
            response = send_message_to_thread(WORKSPACE_SLUG, thread_slug, message_text)
            
            # Captures the model's response, if it exists
            text_response = response.get('textResponse', 'No response generated')
            print(f"Model response: {text_response}")

            # Returns an HTTP 200 response to the sender, including the textResponse field
            self.send_response(200)
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            # Sends the response with the generated text
            self.wfile.write(json.dumps({
                "status": "success",
                "textResponse": text_response  # Sends the model's response along
            }).encode("utf-8"))
            
        except Exception as e:
            print(f"Error: {e}")
            self.send_response(500)
            self.end_headers()
            self.wfile.write(json.dumps({"status": "error", "message": str(e)}).encode("utf-8"))

def run(server_class=HTTPServer, handler_class=WebhookHandler, port=8080):
    """Starts the HTTP server."""
    server_address = ("", port)
    httpd = server_class(server_address, handler_class)
    print(f"Server running on port {port}")
    httpd.serve_forever()

if __name__ == "__main__":
    run()
Originally created by @Peterson047 on GitHub (Dec 10, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2791 ### What would you like to see? Hello everyone! I've been working on a project for months and I've encountered a challenge with session management. I tested a solution using Dialogflow CX and was able to use the JSON from the request to define the session via WhatsApp, where the person's number becomes the thread ID. However, when I tried to implement something similar in AnythingLLM, I couldn't find a ready-made solution for this. To get around the problem, I used the thread creation and message sending APIs. My webhook is hosted on Google Cloud Functions and receives requests via Make (Integromat). Since there is no API to list existing threads, the code handles errors assuming that the thread already exists and, thus, concatenates a link to send the request. My question is: is there a better way to manage sessions in a more optimized way? My solution works well, but the lack of an API to list the threads created in a workspace makes the process difficult. Is the team already working on something along these lines, or is there an alternative approach I can apply to improve session management? I welcome any suggestions! **My code:** ```import json from http.server import BaseHTTPRequestHandler, HTTPServer import requests # AnythingLLM API configurations API_BASE_URL = "http://YOUR_DOMAIN/api/v1/workspace" WORKSPACE_SLUG = "WKP-NAME" API_KEY = "API-ANYTHING" # Simulates an in-memory database to associate wa_id with threads user_threads = {} def create_thread(name, slug): """Creates a new thread in AnythingLLM.""" url = f"{API_BASE_URL}/{WORKSPACE_SLUG}/thread/new" headers = { "accept": "application/json", "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json" } data = { "userId": 1, # This is fixed, can be changed depending on the case "name": name, "slug": slug } response = requests.post(url, headers=headers, json=data) if response.status_code == 200: return response.json()["thread"] else: raise Exception(f"Error creating thread: {response.text}") def send_message_to_thread(workspace_slug, thread_slug, message): """Sends a message to a thread in AnythingLLM.""" url = f"{API_BASE_URL}/{workspace_slug}/thread/{thread_slug}/chat" headers = { "accept": "application/json", "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json" } data = { "message": message, "mode": "chat", # Can be 'chat' or 'query' "userId": 1 # This is fixed, can be changed depending on the case } response = requests.post(url, headers=headers, json=data) if response.status_code == 200: return response.json() else: raise Exception(f"Error sending message: {response.text}") class WebhookHandler(BaseHTTPRequestHandler): def do_POST(self): """Handles incoming POST requests to the webhook.""" content_length = int(self.headers["Content-Length"]) post_data = self.rfile.read(content_length) try: request_body = json.loads(post_data) wa_id = request_body[0]["contacts"][0]["wa_id"] name = request_body[0]["contacts"][0]["profile"]["name"] message_text = request_body[0]["messages"][0]["text"]["body"] # Checks if the user already has a thread if wa_id not in user_threads: try: # Creates a new thread for the user thread = create_thread(name, wa_id) user_threads[wa_id] = thread["slug"] print(f"New thread created for {wa_id}: {thread['slug']}") except Exception as e: print(f"Error creating thread for {wa_id}: {e}") if 'NoneType' in str(e): # If the error is related to 'NoneType' # Assumes the thread already exists and associates it print(f"The thread already exists for {wa_id}, associating with slug") user_threads[wa_id] = wa_id # Uses wa_id as slug to associate with the existing thread else: print(f"Existing thread for {wa_id}: {user_threads[wa_id]}") # Retrieves the associated thread thread_slug = user_threads[wa_id] # Sends the message to the associated thread response = send_message_to_thread(WORKSPACE_SLUG, thread_slug, message_text) # Captures the model's response, if it exists text_response = response.get('textResponse', 'No response generated') print(f"Model response: {text_response}") # Returns an HTTP 200 response to the sender, including the textResponse field self.send_response(200) self.send_header("Content-Type", "application/json") self.end_headers() # Sends the response with the generated text self.wfile.write(json.dumps({ "status": "success", "textResponse": text_response # Sends the model's response along }).encode("utf-8")) except Exception as e: print(f"Error: {e}") self.send_response(500) self.end_headers() self.wfile.write(json.dumps({"status": "error", "message": str(e)}).encode("utf-8")) def run(server_class=HTTPServer, handler_class=WebhookHandler, port=8080): """Starts the HTTP server.""" server_address = ("", port) httpd = server_class(server_address, handler_class) print(f"Server running on port {port}") httpd.serve_forever() if __name__ == "__main__": run() ```
yindo added the enhancementfeature request labels 2026-02-22 18:26:32 -05:00
yindo closed this issue 2026-02-22 18:26:32 -05:00
yindo commented 2026-02-22 18:26:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 11, 2024):

Are you creating threads because you intend to use them in the UI? If not and you simply want workspace-focused chats but to have scoped historys when chatting you should leverage the sessionID property
https://github.com/Mintplex-Labs/anything-llm/blob/a0c5d898f00c1f1af8a851e05606680946a91084/server/endpoints/api/workspace/index.js#L588

No need to create threads anymore! If you send a chat with a sessionId of 1234 every chat to this workspace slug with the same sessionId: 1234 will use the same history, it persists forever but does not ever appear in the UI.

Workspace thread creation & management is best used for automatic thread creation that will be later used in the app UI, if you dont plan on accessing the chats in the UI - you can use sessionId and its way easier to manage.

In this example, sessionId would probably be the whatsapp number.

Let me know if that helps or if I need to reopen this as I could have totally missed the point

@timothycarambat commented on GitHub (Dec 11, 2024): Are you creating threads because you intend to use them in the UI? If not and you simply want workspace-focused chats but to have scoped historys when chatting you should leverage the `sessionID` property https://github.com/Mintplex-Labs/anything-llm/blob/a0c5d898f00c1f1af8a851e05606680946a91084/server/endpoints/api/workspace/index.js#L588 No need to create threads anymore! If you send a chat with a `sessionId` of `1234` every chat to this workspace slug with the same `sessionId: 1234` will use the same history, it persists forever but does not ever appear in the UI. Workspace thread creation & management is best used for automatic thread creation that will be later used in the app UI, if you dont plan on accessing the chats in the UI - you can use sessionId and its way easier to manage. In this example, sessionId would probably be the whatsapp number. Let me know if that helps or if I need to reopen this as I could have totally missed the point
yindo commented 2026-02-22 18:26:32 -05:00
Author
Owner
Copy Link
Copy Source

@Peterson047 commented on GitHub (Dec 11, 2024):

I came across this issue in older versions. Back then, files in RAG weren't considered in requests when using IDs. I opted for threads because the responses were based on the workspace documents. Do you know if this behavior with IDs has been fixed? If I create sessions by ID, will the responses be based on the RAG files to generate the output?

@Peterson047 commented on GitHub (Dec 11, 2024): I came across this issue in older versions. Back then, files in RAG weren't considered in requests when using IDs. I opted for threads because the responses were based on the workspace documents. Do you know if this behavior with IDs has been fixed? If I create sessions by ID, will the responses be based on the RAG files to generate the output?
yindo commented 2026-02-22 18:26:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 11, 2024):

The sessionId basically emulates a thread without having to create one. Its still under a workspace, so RAG works the exact same and your chat will still use the workspace's documents all the same

@timothycarambat commented on GitHub (Dec 11, 2024): The sessionId basically emulates a thread without having to create one. Its still under a workspace, so RAG works the exact same and your chat will still use the workspace's documents all the same
yindo commented 2026-02-22 18:26:33 -05:00
Author
Owner
Copy Link
Copy Source

@Peterson047 commented on GitHub (Dec 12, 2024):

I will test it then and provide feedback. Could you explain better the difference between:

/v1/workspace/{slug}/chat
and:
/v1/workspace/{slug}/stream-chat ?

@Peterson047 commented on GitHub (Dec 12, 2024): I will test it then and provide feedback. Could you explain better the **difference between:** `/v1/workspace/{slug}/chat` **and:** `/v1/workspace/{slug}/stream-chat` ?
yindo commented 2026-02-22 18:26:33 -05:00
Author
Owner
Copy Link
Copy Source

@Peterson047 commented on GitHub (Dec 12, 2024):

I just tested both endpoints, and they didn't return the correct response as when I create a pinned thread. It seems like the issue with the old versions persists, as they don't take pinned documents into account.

Edit:* I identified the error; it turns out that the ID I was using only contained numbers. For some reason, this didn't satisfy a requirement for consideration in the RAG.

@Peterson047 commented on GitHub (Dec 12, 2024): I just tested both endpoints, and they didn't return the correct response as when I create a pinned thread. It seems like the issue with the old versions persists, as they don't take pinned documents into account. **Edit*:** I identified the error; it turns out that the ID I was using only contained numbers. For some reason, this didn't satisfy a requirement for consideration in the RAG.
yindo commented 2026-02-22 18:26:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 12, 2024):

Edit:* I identified the error; it turns out that the ID I was using only contained numbers. For some reason, this didn't satisfy a requirement for consideration in the RAG.

That is very weird, we just cast to a String
https://github.com/Mintplex-Labs/anything-llm/blob/2e8f9d8c0856707b5da12ef56e531564eb134030/server/endpoints/api/workspace/index.js#L665

some examples

sessionId = 0123418121812 // leading zero
!!sessionId ? String(sessionId) : null // '123418121812'

sessionId =123
!!sessionId ? String(sessionId) : null // '123'

sessionId = 0
!!sessionId ? String(sessionId) : null //  null - this would be unexpected
@timothycarambat commented on GitHub (Dec 12, 2024): > Edit:* I identified the error; it turns out that the ID I was using only contained numbers. For some reason, this didn't satisfy a requirement for consideration in the RAG. That is very weird, we just cast to a String https://github.com/Mintplex-Labs/anything-llm/blob/2e8f9d8c0856707b5da12ef56e531564eb134030/server/endpoints/api/workspace/index.js#L665 some examples ```javascript sessionId = 0123418121812 // leading zero !!sessionId ? String(sessionId) : null // '123418121812' sessionId =123 !!sessionId ? String(sessionId) : null // '123' sessionId = 0 !!sessionId ? String(sessionId) : null // null - this would be unexpected ```
yindo changed title from [FEAT]: Session Management with AnythingLLM and API to [GH-ISSUE #2791] [FEAT]: Session Management with AnythingLLM and API 2026-06-05 14:42:41 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1788
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?