Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2091] Issue with Managing Conversation Context in AnythingLLM Using threadSlug #1362

New Issue
Closed
opened 2026-02-22 18:24:26 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:24:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @G3kPLAY on GitHub (Aug 11, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2091

Originally assigned to: @shatfield4 on GitHub.

I’ve been working on a project where I’m integrating AnythingLLM with a custom WebUI that interacts with a specialized SQL assistant model (LLaMA 3.1). The setup is primarily used for generating and optimizing SQL queries in Teradata. We’re using AnythingLLM to manage different conversation threads with the model.

Current Setup:
WebUI: A custom front-end that allows users to interact with AnythingLLM.
Backend: Flask application that connects the WebUI to AnythingLLM, handling the API requests.
Model: LLaMA 3.1 model, integrated with AnythingLLM for natural language query optimization in Teradata.
The Problem:
Every time a new conversation is started, a new threadSlug is generated and passed from the WebUI to AnythingLLM. Despite correctly generating and using new threadSlugs, the responses from the model appear to be carrying over context from previous conversations.

Code Snippets:
Here’s a summary of how we’re handling conversations and threadSlugs in our routes.py:

import requests
from flask import request, jsonify
import logging
import uuid

API_KEY = "YOUR_API_KEY_HERE"  # Placeholder for the actual API key
THREAD_URL = "http://localhost:3001/api/v1/workspace/taisa/thread/new"
DELETE_THREAD_URL = "http://localhost:3001/api/v1/workspace/taisa/thread/"
CHAT_URL = "http://localhost:3001/api/v1/workspace/taisa/chat"

conversation_id = None  # Variable to store the current threadSlug

def start_new_conversation():
    global conversation_id
    previous_conversation_id = conversation_id
    conversation_id = str(uuid.uuid4())

    try:
        response = requests.post(THREAD_URL, headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json"
        }, json={"slug": conversation_id})

        response.raise_for_status()
        data = response.json()

        if "thread" in data and "slug" in data["thread"]:
            conversation_id = data["thread"]["slug"]
            logging.debug(f'New conversation automatically started with ID: {conversation_id}')

            # Delete the previous thread if it exists
            if previous_conversation_id:
                delete_thread(previous_conversation_id)

        else:
            logging.error('Failed to start a new conversation: No conversation ID returned from API.')
            conversation_id = None

    except requests.exceptions.RequestException as e:
        logging.error(f'Error starting a new conversation with the API: {e}')
        conversation_id = None

def delete_thread(thread_slug):
    try:
        delete_url = f"{DELETE_THREAD_URL}{thread_slug}"
        response = requests.delete(delete_url, headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json"
        })

        if response.status_code == 200:
            logging.debug(f'Thread {thread_slug} successfully deleted.')
        else:
            logging.error(f'Failed to delete thread {thread_slug}. Status code: {response.status_code}')

    except requests.exceptions.RequestException as e:
        logging.error(f'Error deleting thread {thread_slug}: {e}')

@app.route('/ask', methods=['POST'])
def ask():
    global conversation_id
    user_message = request.json.get('message')
    received_thread_slug = request.json.get('thread_slug')

    logging.debug(f'Received message: {user_message}')
    logging.debug(f'Received thread_slug: {received_thread_slug}')

    payload = {
        "message": user_message,
        "mode": "chat",
        "thread_slug": received_thread_slug
    }

    try:
        response = requests.post(CHAT_URL, headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json"
        }, json=payload)

        response.raise_for_status()
        data = response.json()

        text_response = data.get('textResponse')
        if text_response is None:
            logging.error('No response received from the model or there was an error with the API.')
            text_response = 'Error: Could not retrieve a response from the model.'

    except requests.exceptions.RequestException as e:
        logging.error(f'Error communicating with the API: {e}')
        text_response = 'Error: Could not connect to the API.'

    logging.debug(f'Response from API: {text_response}')

    formatted_response = text_response.replace('\\n', '<br>')
    return {"response": formatted_response}

@app.route('/new_conversation', methods=['POST'])
def new_conversation():
    start_new_conversation()
    if conversation_id:
        return jsonify({"status": "New conversation started.", "conversation_id": conversation_id})
    else:
        return jsonify({"status": "Failed to start new conversation."}), 500

Observations:
threadSlug Generation: The threadSlugs are correctly generated and appear to be unique.
Context Carryover: Despite deleting old threads and ensuring new threadSlugs are used, the model seems to retain context from previous conversations, affecting the responses.

What We Need:
Guidance on Ensuring Separation of Context: How can we ensure that each new threadSlug completely isolates the conversation from any previous context?
Proper API Usage: Are there any additional steps or endpoints we should be utilizing to ensure that old contexts do not persist when starting a new conversation?
Best Practices: Any insights or best practices on managing conversation context in AnythingLLM, particularly when using threadSlugs.

Thank you in advance for your help!

Originally created by @G3kPLAY on GitHub (Aug 11, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2091 Originally assigned to: @shatfield4 on GitHub. I’ve been working on a project where I’m integrating AnythingLLM with a custom WebUI that interacts with a specialized SQL assistant model (LLaMA 3.1). The setup is primarily used for generating and optimizing SQL queries in Teradata. We’re using AnythingLLM to manage different conversation threads with the model. Current Setup: WebUI: A custom front-end that allows users to interact with AnythingLLM. Backend: Flask application that connects the WebUI to AnythingLLM, handling the API requests. Model: LLaMA 3.1 model, integrated with AnythingLLM for natural language query optimization in Teradata. The Problem: Every time a new conversation is started, a new threadSlug is generated and passed from the WebUI to AnythingLLM. Despite correctly generating and using new threadSlugs, the responses from the model appear to be carrying over context from previous conversations. Code Snippets: Here’s a summary of how we’re handling conversations and threadSlugs in our routes.py: ``` import requests from flask import request, jsonify import logging import uuid API_KEY = "YOUR_API_KEY_HERE" # Placeholder for the actual API key THREAD_URL = "http://localhost:3001/api/v1/workspace/taisa/thread/new" DELETE_THREAD_URL = "http://localhost:3001/api/v1/workspace/taisa/thread/" CHAT_URL = "http://localhost:3001/api/v1/workspace/taisa/chat" conversation_id = None # Variable to store the current threadSlug def start_new_conversation(): global conversation_id previous_conversation_id = conversation_id conversation_id = str(uuid.uuid4()) try: response = requests.post(THREAD_URL, headers={ "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json" }, json={"slug": conversation_id}) response.raise_for_status() data = response.json() if "thread" in data and "slug" in data["thread"]: conversation_id = data["thread"]["slug"] logging.debug(f'New conversation automatically started with ID: {conversation_id}') # Delete the previous thread if it exists if previous_conversation_id: delete_thread(previous_conversation_id) else: logging.error('Failed to start a new conversation: No conversation ID returned from API.') conversation_id = None except requests.exceptions.RequestException as e: logging.error(f'Error starting a new conversation with the API: {e}') conversation_id = None def delete_thread(thread_slug): try: delete_url = f"{DELETE_THREAD_URL}{thread_slug}" response = requests.delete(delete_url, headers={ "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json" }) if response.status_code == 200: logging.debug(f'Thread {thread_slug} successfully deleted.') else: logging.error(f'Failed to delete thread {thread_slug}. Status code: {response.status_code}') except requests.exceptions.RequestException as e: logging.error(f'Error deleting thread {thread_slug}: {e}') @app.route('/ask', methods=['POST']) def ask(): global conversation_id user_message = request.json.get('message') received_thread_slug = request.json.get('thread_slug') logging.debug(f'Received message: {user_message}') logging.debug(f'Received thread_slug: {received_thread_slug}') payload = { "message": user_message, "mode": "chat", "thread_slug": received_thread_slug } try: response = requests.post(CHAT_URL, headers={ "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json" }, json=payload) response.raise_for_status() data = response.json() text_response = data.get('textResponse') if text_response is None: logging.error('No response received from the model or there was an error with the API.') text_response = 'Error: Could not retrieve a response from the model.' except requests.exceptions.RequestException as e: logging.error(f'Error communicating with the API: {e}') text_response = 'Error: Could not connect to the API.' logging.debug(f'Response from API: {text_response}') formatted_response = text_response.replace('\\n', '<br>') return {"response": formatted_response} @app.route('/new_conversation', methods=['POST']) def new_conversation(): start_new_conversation() if conversation_id: return jsonify({"status": "New conversation started.", "conversation_id": conversation_id}) else: return jsonify({"status": "Failed to start new conversation."}), 500 ``` Observations: threadSlug Generation: The threadSlugs are correctly generated and appear to be unique. Context Carryover: Despite deleting old threads and ensuring new threadSlugs are used, the model seems to retain context from previous conversations, affecting the responses. What We Need: Guidance on Ensuring Separation of Context: How can we ensure that each new threadSlug completely isolates the conversation from any previous context? Proper API Usage: Are there any additional steps or endpoints we should be utilizing to ensure that old contexts do not persist when starting a new conversation? Best Practices: Any insights or best practices on managing conversation context in AnythingLLM, particularly when using threadSlugs. Thank you in advance for your help!
yindo added the possible buginvestigating labels 2026-02-22 18:24:26 -05:00
yindo closed this issue 2026-02-22 18:24:26 -05:00
yindo commented 2026-02-22 18:24:26 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Aug 12, 2024):

@G3kPLAY I see your issue here. The context is being carried over because you are chatting with just the default thread.

You are using CHAT_URL = "http://localhost:3001/api/v1/workspace/{slug}/chat" as the endpoint for chatting. Using this endpoint will chat with the default thread on that workspace.

You need to use the http://localhost:3001/v1/workspace/{slug}/thread/{threadSlug}/chat endpoint to chat directly with a thread.

@shatfield4 commented on GitHub (Aug 12, 2024): @G3kPLAY I see your issue here. The context is being carried over because you are chatting with just the default thread. You are using `CHAT_URL = "http://localhost:3001/api/v1/workspace/{slug}/chat"` as the endpoint for chatting. Using this endpoint will chat with the default thread on that workspace. You need to use the `http://localhost:3001/v1/workspace/{slug}/thread/{threadSlug}/chat` endpoint to chat directly with a thread.
yindo changed title from Issue with Managing Conversation Context in AnythingLLM Using threadSlug to [GH-ISSUE #2091] Issue with Managing Conversation Context in AnythingLLM Using threadSlug 2026-06-05 14:40:21 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1362
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?