Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2278] [Feat]: Custom skill calling via API agent #1487

New Issue
Closed
opened 2026-02-22 18:25:04 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:25:04 -05:00
Owner
Copy Link
Copy Source

Originally created by @daaddd on GitHub (Sep 12, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2278

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

Custom agent skills are not always triggering.

model2 calls work fine and invoke the agent successfully, model1 outputs agent invocation in text but doesnt call it, is there a way to force @agent calls? why is it not seem/perceived by the llm?

app.py

from flask import Flask # type: ignore
import logging
from routes.model1 import model1_webhook
from routes.model2 import model2_webhook

# Initialize the Flask app
app = Flask(__name__)

# Configure logging
logging.basicConfig(level=logging.DEBUG)

# Register routes
@app.route("/webhook/model1", methods=["POST"])
def model1():
    return model1_webhook()

@app.route("/webhook/model2", methods=["POST"])
def model2():
    return model2_webhook()

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

utils.py

import requests # type: ignore

def call_custom_llm_aws_api(input_text, system_prompt, model_name):
    url = "https://myawsdomain.com/api/v1/openai/chat/completions"
    headers = {
        "accept": "application/json",
        "Content-Type": "application/json",
        "Authorization": "Bearer 7XN..blaaa..1VHDF81"
    }
    data = {
        "model": model_name,
        "messages": [
            {
                "role": "system",
                "content": system_prompt
            },
            {
                "role": "user",
                "content": input_text
            }
        ]
    }

    response = requests.post(url, headers=headers, json=data)
    
    if response.status_code == 200:
        return response.json().get("choices", [{}])[0].get("message", {}).get("content", "No response from model")
    else:
        return f"Error: {response.status_code}, {response.text}"

model1.py

import sys
import json
from flask import request
from twilio.twiml.messaging_response import MessagingResponse
from utils import call_custom_llm_aws_api
from datetime import datetime, timedelta

def model1_webhook():
    input_text = request.form.get("Body", "")

    print('----------------------', file=sys.stderr)
    print(f"Input: {input_text}", file=sys.stderr)
    print('----------------------', file=sys.stderr)

    system_prompt = """You are an AI assistant that helps schedule meetings. When a user asks to schedule a meeting, extract the necessary details (title, start time, end time, attendees) and use the Schedule_Google_Calendar_Event agent to create the event.

    To schedule a meeting, use the following command format:
    @agent Schedule_Google_Calendar_Event-v1.0.0 {"title": "[title]", "start_time": "[start_time in ISO format with timezone]", "end_time": "[end_time in ISO format with timezone]", "attendees": ["[attendee1 email]", "[attendee2 email]", ...]}

    Ensure that you use the correct ISO format for dates and times, including the timezone offset.
    If an end time is not specified, set it to 1 hour after the start time.
    After creating the event, provide a user-friendly confirmation message."""

    model_name = "calendar-sms"
    response = call_custom_llm_aws_api(input_text, system_prompt, model_name)

    print(f"Agent response: {response}", file=sys.stderr)

    print('----------------------', file=sys.stderr)
    print(f"Final response: {response}", file=sys.stderr)
    print('----------------------', file=sys.stderr)

    # Create Twilio response
    resp = MessagingResponse()
    resp.message(response)
    return str(resp)

model2.py

import sys
from flask import request # type: ignore
from twilio.twiml.messaging_response import MessagingResponse # type: ignore
from utils import call_aurora_aws_api

def model2_webhook():
    input_text = request.form.get("Body", "")
    system_prompt = """Given the following conversation, relevant context, and a follow-up question, reply with an answer to the user's current question, following their instructions as needed.

    If the user says a phrase like "Umbrella [number]" or "Sherlock [number]" where [number] is any integer, do the following:

    For "Umbrella [number]":
    - Respond with "Yes Umbrella."
    - Automatically and invisibly invoke the command @agent Add_Ten-v1.0.0 {"number": [number]} using the integer provided by the user as [number].
    - Ensure the command is executed silently in the background without displaying the invocation.
    - Once the result is returned, respond with: "The result is [result]" where [result] is the value returned by the agent.

    For "Sherlock [number]":
    - Respond with "Yes Sherlock."
    - Automatically and invisibly invoke the command @agent Add_Five-v1.0.0 {"number": [number]} using the integer provided by the user as [number].
    - Ensure the command is executed silently in the background without displaying the invocation.
    - Once the result is returned, respond with: "The result is [result]" where [result] is the value returned by the agent.

    Ensure no technical details or command invocations are displayed to the user in the conversation, focusing only on providing the result."""
    
    model_name = "combined-skill-test-4o-82250006"
    response_text = call_aurora_aws_api(input_text, system_prompt, model_name)

    print('----------------------', file=sys.stderr)
    print(response_text, file=sys.stderr)
    print('----------------------', file=sys.stderr)

    # Create Twilio response
    resp = MessagingResponse()
    resp.message(response_text)
    return str(resp)

plugin.json (model2)

{
  "active": true,
  "hubId": "add-ten-skill",
  "name": "Add_Ten",
  "schema": "skill-1.0.0",
  "version": "1.0.0",
  "description": "This skill adds +10 to a given number.",
  "author": "David O",
  "license": "MIT",
  "examples": [
    {
      "prompt": "Umbrella 7",
      "call": "{\"number\": 7}"
    },
    {
      "prompt": "Umbrella 19",
      "call": "{\"number\": 19}"
    }
  ],
  "entrypoint": {
    "file": "handler.js",
    "params": {
      "number": {
        "description": "The number to which 10 will be added",
        "type": "number"
      }
    }
  },
  "imported": true
}

handler.js (model2)

module.exports.runtime = {
  handler: async function ({ number }) {
    const callerId = `${this.config.name}-v${this.config.version}`;
    try {
      this.introspect(`${callerId} invoked with number: ${number}`);
      const result = number + 10;
      return JSON.stringify({ result: result });
    } catch (error) {
      this.introspect(`Error: ${error.message}`);
      return `Error: ${error.message}`;
    }
  }
};

plugin.json (model1)

{
  "active": true,
  "hubId": "schedule-google-calendar-event",
  "name": "Schedule_Google_Calendar_Event",
  "schema": "skill-1.0.0",
  "version": "1.0.0",
  "description": "Schedules an event on Google Calendar using a Zapier Webhook",
  "author": "David Oliveira",
  "license": "MIT",
  "examples": [
    {
      "prompt": "Schedule a meeting with John at 3pm tomorrow",
      "call": "{\"title\": \"Meeting with John\", \"start_time\": \"2024-09-13T15:00:00\", \"end_time\": \"2024-09-13T16:00:00\", \"attendees\": [\"john@example.com\"]}"
    },
    {
      "prompt": "Book a team sync for 2pm today",
      "call": "{\"title\": \"Team Sync\", \"start_time\": \"2024-09-12T14:00:00\", \"end_time\": \"2024-09-12T15:00:00\", \"attendees\": [\"team@example.com\"]}"
    }
  ],
  "entrypoint": {
    "file": "handler.js",
    "params": {
      "title": {
        "description": "Title of the event",
        "type": "string"
      },
      "start_time": {
        "description": "Start time of the event",
        "type": "string"
      },
      "end_time": {
        "description": "End time of the event",
        "type": "string"
      },
      "attendees": {
        "description": "List of attendee email addresses",
        "type": "array",
        "items": {
          "type": "string"
        }
      }
    }
  },
  "imported": true
}

handler.js (model1)

const fetch = require('node-fetch');

module.exports.runtime = {
  handler: async function ({ title, start_time, end_time, attendees }) {
    const zapierWebhookUrl = 'https://hooks.zapier.com/hooks/catch/48...975/2h....md/';
    const payload = {
      title,
      start_time,
      end_time,
      attendees
    };
    try {
      const response = await fetch(zapierWebhookUrl, {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify(payload),
      });
      const data = await response.json();
      if (!response.ok) {
        throw new Error(data.error || 'Failed to create event');
      }
      return JSON.stringify({ message: 'Event scheduled successfully', event: data });
    } catch (e) {
      this.introspect(`Error scheduling event: ${e.message}`);
      return JSON.stringify({ error: `Failed to schedule event: ${e.message}` });
    }
  },
};

finally calling the command directly in chat works:

Screenshot 2024-09-12 at 7 35 18 PM

but not when using the webhook, chat transcript:

2024-09-12 19:28:37  * Serving Flask app 'app'
2024-09-12 19:28:47 ----------------------
2024-09-12 19:28:47 Input: Hi there, book meeting 9pm ET today to talk about the sms project, david daaddd@gmail.com
2024-09-12 19:28:47 ----------------------
2024-09-12 19:28:47 DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): mydomain.com:443
2024-09-12 19:28:52 DEBUG:urllib3.connectionpool:https://mydomain.com:443 "POST /api/v1/openai/chat/completions HTTP/11" 200 1037
2024-09-12 19:28:52 Agent response: Sure, I'll schedule the meeting.
2024-09-12 19:28:52 
2024-09-12 19:28:52 Assuming the meeting is scheduled for today at 9 PM ET, the details are:
2024-09-12 19:28:52 
2024-09-12 19:28:52 - **Title**: SMS Project Discussion
2024-09-12 19:28:52 - **Start Time**: 9 PM ET (Eastern Time) today (will be converted to ISO format)
2024-09-12 19:28:52 - **End Time**: 10 PM ET (Eastern Time) today (1 hour after the start time)
2024-09-12 19:28:52 - **Attendee**: david daaddd@gmail.com
2024-09-12 19:28:52 
2024-09-12 19:28:52 Let's first convert the start and end times to ISO format. If today is October 5, 2023, then:
2024-09-12 19:28:52 
2024-09-12 19:28:52 - Start Time in ISO Format: 2023-10-05T21:00:00-04:00
2024-09-12 19:28:52 - End Time in ISO Format: 2023-10-05T22:00:00-04:00
2024-09-12 19:28:52 
2024-09-12 19:28:52 Now, I'll schedule the meeting:
2024-09-12 19:28:52 
2024-09-12 19:28:52 @agent Schedule_Google_Calendar_Event-v1.0.0 {"title": "SMS Project Discussion", "start_time": "2023-10-05T21:00:00-04:00", "end_time": "2023-10-05T22:00:00-04:00", "attendees": ["daaddd@gmail.com"]}
2024-09-12 19:28:52 ----------------------
2024-09-12 19:28:52 Final response: Sure, I'll schedule the meeting.
2024-09-12 19:28:52 
2024-09-12 19:28:52 Assuming the meeting is scheduled for today at 9 PM ET, the details are:
2024-09-12 19:28:52 
2024-09-12 19:28:52 - **Title**: SMS Project Discussion
2024-09-12 19:28:52 - **Start Time**: 9 PM ET (Eastern Time) today (will be converted to ISO format)
2024-09-12 19:28:52 - **End Time**: 10 PM ET (Eastern Time) today (1 hour after the start time)
2024-09-12 19:28:52 - **Attendee**: david daaddd@gmail.com
2024-09-12 19:28:52 
2024-09-12 19:28:52 Let's first convert the start and end times to ISO format. If today is October 5, 2023, then:
2024-09-12 19:28:52 
2024-09-12 19:28:52 - Start Time in ISO Format: 2023-10-05T21:00:00-04:00
2024-09-12 19:28:52 - End Time in ISO Format: 2023-10-05T22:00:00-04:00
2024-09-12 19:28:52 
2024-09-12 19:28:52 Now, I'll schedule the meeting:
2024-09-12 19:28:52 
2024-09-12 19:28:52 @agent Schedule_Google_Calendar_Event-v1.0.0 {"title": "SMS Project Discussion", "start_time": "2023-10-05T21:00:00-04:00", "end_time": "2023-10-05T22:00:00-04:00", "attendees": ["daaddd@gmail.com"]}
2024-09-12 19:28:52 ----------------------
2024-09-12 19:28:52 INFO:werkzeug:192.168.65.1 - - [12/Sep/2024 23:28:52] "POST /webhook/model1 HTTP/1.1" 200 -

Are there known steps to reproduce?

No response

Originally created by @daaddd on GitHub (Sep 12, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2278 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? Custom agent skills are not always triggering. `model2` calls work fine and invoke the agent successfully, `model1` outputs agent invocation in text but doesnt call it, is there a way to force @agent calls? why is it not seem/perceived by the llm? app.py ``` from flask import Flask # type: ignore import logging from routes.model1 import model1_webhook from routes.model2 import model2_webhook # Initialize the Flask app app = Flask(__name__) # Configure logging logging.basicConfig(level=logging.DEBUG) # Register routes @app.route("/webhook/model1", methods=["POST"]) def model1(): return model1_webhook() @app.route("/webhook/model2", methods=["POST"]) def model2(): return model2_webhook() if __name__ == "__main__": app.run(host="0.0.0.0", port=5000) ``` utils.py ``` import requests # type: ignore def call_custom_llm_aws_api(input_text, system_prompt, model_name): url = "https://myawsdomain.com/api/v1/openai/chat/completions" headers = { "accept": "application/json", "Content-Type": "application/json", "Authorization": "Bearer 7XN..blaaa..1VHDF81" } data = { "model": model_name, "messages": [ { "role": "system", "content": system_prompt }, { "role": "user", "content": input_text } ] } response = requests.post(url, headers=headers, json=data) if response.status_code == 200: return response.json().get("choices", [{}])[0].get("message", {}).get("content", "No response from model") else: return f"Error: {response.status_code}, {response.text}" ``` model1.py ``` import sys import json from flask import request from twilio.twiml.messaging_response import MessagingResponse from utils import call_custom_llm_aws_api from datetime import datetime, timedelta def model1_webhook(): input_text = request.form.get("Body", "") print('----------------------', file=sys.stderr) print(f"Input: {input_text}", file=sys.stderr) print('----------------------', file=sys.stderr) system_prompt = """You are an AI assistant that helps schedule meetings. When a user asks to schedule a meeting, extract the necessary details (title, start time, end time, attendees) and use the Schedule_Google_Calendar_Event agent to create the event. To schedule a meeting, use the following command format: @agent Schedule_Google_Calendar_Event-v1.0.0 {"title": "[title]", "start_time": "[start_time in ISO format with timezone]", "end_time": "[end_time in ISO format with timezone]", "attendees": ["[attendee1 email]", "[attendee2 email]", ...]} Ensure that you use the correct ISO format for dates and times, including the timezone offset. If an end time is not specified, set it to 1 hour after the start time. After creating the event, provide a user-friendly confirmation message.""" model_name = "calendar-sms" response = call_custom_llm_aws_api(input_text, system_prompt, model_name) print(f"Agent response: {response}", file=sys.stderr) print('----------------------', file=sys.stderr) print(f"Final response: {response}", file=sys.stderr) print('----------------------', file=sys.stderr) # Create Twilio response resp = MessagingResponse() resp.message(response) return str(resp) ``` model2.py ``` import sys from flask import request # type: ignore from twilio.twiml.messaging_response import MessagingResponse # type: ignore from utils import call_aurora_aws_api def model2_webhook(): input_text = request.form.get("Body", "") system_prompt = """Given the following conversation, relevant context, and a follow-up question, reply with an answer to the user's current question, following their instructions as needed. If the user says a phrase like "Umbrella [number]" or "Sherlock [number]" where [number] is any integer, do the following: For "Umbrella [number]": - Respond with "Yes Umbrella." - Automatically and invisibly invoke the command @agent Add_Ten-v1.0.0 {"number": [number]} using the integer provided by the user as [number]. - Ensure the command is executed silently in the background without displaying the invocation. - Once the result is returned, respond with: "The result is [result]" where [result] is the value returned by the agent. For "Sherlock [number]": - Respond with "Yes Sherlock." - Automatically and invisibly invoke the command @agent Add_Five-v1.0.0 {"number": [number]} using the integer provided by the user as [number]. - Ensure the command is executed silently in the background without displaying the invocation. - Once the result is returned, respond with: "The result is [result]" where [result] is the value returned by the agent. Ensure no technical details or command invocations are displayed to the user in the conversation, focusing only on providing the result.""" model_name = "combined-skill-test-4o-82250006" response_text = call_aurora_aws_api(input_text, system_prompt, model_name) print('----------------------', file=sys.stderr) print(response_text, file=sys.stderr) print('----------------------', file=sys.stderr) # Create Twilio response resp = MessagingResponse() resp.message(response_text) return str(resp) ``` plugin.json (model2) ``` { "active": true, "hubId": "add-ten-skill", "name": "Add_Ten", "schema": "skill-1.0.0", "version": "1.0.0", "description": "This skill adds +10 to a given number.", "author": "David O", "license": "MIT", "examples": [ { "prompt": "Umbrella 7", "call": "{\"number\": 7}" }, { "prompt": "Umbrella 19", "call": "{\"number\": 19}" } ], "entrypoint": { "file": "handler.js", "params": { "number": { "description": "The number to which 10 will be added", "type": "number" } } }, "imported": true } ``` handler.js (model2) ``` module.exports.runtime = { handler: async function ({ number }) { const callerId = `${this.config.name}-v${this.config.version}`; try { this.introspect(`${callerId} invoked with number: ${number}`); const result = number + 10; return JSON.stringify({ result: result }); } catch (error) { this.introspect(`Error: ${error.message}`); return `Error: ${error.message}`; } } }; ``` plugin.json (model1) ``` { "active": true, "hubId": "schedule-google-calendar-event", "name": "Schedule_Google_Calendar_Event", "schema": "skill-1.0.0", "version": "1.0.0", "description": "Schedules an event on Google Calendar using a Zapier Webhook", "author": "David Oliveira", "license": "MIT", "examples": [ { "prompt": "Schedule a meeting with John at 3pm tomorrow", "call": "{\"title\": \"Meeting with John\", \"start_time\": \"2024-09-13T15:00:00\", \"end_time\": \"2024-09-13T16:00:00\", \"attendees\": [\"john@example.com\"]}" }, { "prompt": "Book a team sync for 2pm today", "call": "{\"title\": \"Team Sync\", \"start_time\": \"2024-09-12T14:00:00\", \"end_time\": \"2024-09-12T15:00:00\", \"attendees\": [\"team@example.com\"]}" } ], "entrypoint": { "file": "handler.js", "params": { "title": { "description": "Title of the event", "type": "string" }, "start_time": { "description": "Start time of the event", "type": "string" }, "end_time": { "description": "End time of the event", "type": "string" }, "attendees": { "description": "List of attendee email addresses", "type": "array", "items": { "type": "string" } } } }, "imported": true } ``` handler.js (model1) ``` const fetch = require('node-fetch'); module.exports.runtime = { handler: async function ({ title, start_time, end_time, attendees }) { const zapierWebhookUrl = 'https://hooks.zapier.com/hooks/catch/48...975/2h....md/'; const payload = { title, start_time, end_time, attendees }; try { const response = await fetch(zapierWebhookUrl, { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(payload), }); const data = await response.json(); if (!response.ok) { throw new Error(data.error || 'Failed to create event'); } return JSON.stringify({ message: 'Event scheduled successfully', event: data }); } catch (e) { this.introspect(`Error scheduling event: ${e.message}`); return JSON.stringify({ error: `Failed to schedule event: ${e.message}` }); } }, }; ``` finally calling the command directly in chat works: <img width="1612" alt="Screenshot 2024-09-12 at 7 35 18 PM" src="https://github.com/user-attachments/assets/2c22858c-41aa-41a2-94a2-dd5fc5f60163"> but not when using the webhook, chat transcript: ``` 2024-09-12 19:28:37 * Serving Flask app 'app' 2024-09-12 19:28:47 ---------------------- 2024-09-12 19:28:47 Input: Hi there, book meeting 9pm ET today to talk about the sms project, david daaddd@gmail.com 2024-09-12 19:28:47 ---------------------- 2024-09-12 19:28:47 DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): mydomain.com:443 2024-09-12 19:28:52 DEBUG:urllib3.connectionpool:https://mydomain.com:443 "POST /api/v1/openai/chat/completions HTTP/11" 200 1037 2024-09-12 19:28:52 Agent response: Sure, I'll schedule the meeting. 2024-09-12 19:28:52 2024-09-12 19:28:52 Assuming the meeting is scheduled for today at 9 PM ET, the details are: 2024-09-12 19:28:52 2024-09-12 19:28:52 - **Title**: SMS Project Discussion 2024-09-12 19:28:52 - **Start Time**: 9 PM ET (Eastern Time) today (will be converted to ISO format) 2024-09-12 19:28:52 - **End Time**: 10 PM ET (Eastern Time) today (1 hour after the start time) 2024-09-12 19:28:52 - **Attendee**: david daaddd@gmail.com 2024-09-12 19:28:52 2024-09-12 19:28:52 Let's first convert the start and end times to ISO format. If today is October 5, 2023, then: 2024-09-12 19:28:52 2024-09-12 19:28:52 - Start Time in ISO Format: 2023-10-05T21:00:00-04:00 2024-09-12 19:28:52 - End Time in ISO Format: 2023-10-05T22:00:00-04:00 2024-09-12 19:28:52 2024-09-12 19:28:52 Now, I'll schedule the meeting: 2024-09-12 19:28:52 2024-09-12 19:28:52 @agent Schedule_Google_Calendar_Event-v1.0.0 {"title": "SMS Project Discussion", "start_time": "2023-10-05T21:00:00-04:00", "end_time": "2023-10-05T22:00:00-04:00", "attendees": ["daaddd@gmail.com"]} 2024-09-12 19:28:52 ---------------------- 2024-09-12 19:28:52 Final response: Sure, I'll schedule the meeting. 2024-09-12 19:28:52 2024-09-12 19:28:52 Assuming the meeting is scheduled for today at 9 PM ET, the details are: 2024-09-12 19:28:52 2024-09-12 19:28:52 - **Title**: SMS Project Discussion 2024-09-12 19:28:52 - **Start Time**: 9 PM ET (Eastern Time) today (will be converted to ISO format) 2024-09-12 19:28:52 - **End Time**: 10 PM ET (Eastern Time) today (1 hour after the start time) 2024-09-12 19:28:52 - **Attendee**: david daaddd@gmail.com 2024-09-12 19:28:52 2024-09-12 19:28:52 Let's first convert the start and end times to ISO format. If today is October 5, 2023, then: 2024-09-12 19:28:52 2024-09-12 19:28:52 - Start Time in ISO Format: 2023-10-05T21:00:00-04:00 2024-09-12 19:28:52 - End Time in ISO Format: 2023-10-05T22:00:00-04:00 2024-09-12 19:28:52 2024-09-12 19:28:52 Now, I'll schedule the meeting: 2024-09-12 19:28:52 2024-09-12 19:28:52 @agent Schedule_Google_Calendar_Event-v1.0.0 {"title": "SMS Project Discussion", "start_time": "2023-10-05T21:00:00-04:00", "end_time": "2023-10-05T22:00:00-04:00", "attendees": ["daaddd@gmail.com"]} 2024-09-12 19:28:52 ---------------------- 2024-09-12 19:28:52 INFO:werkzeug:192.168.65.1 - - [12/Sep/2024 23:28:52] "POST /webhook/model1 HTTP/1.1" 200 - ``` ### Are there known steps to reproduce? _No response_
yindo added the enhancementfeature request labels 2026-02-22 18:25:04 -05:00
yindo closed this issue 2026-02-22 18:25:04 -05:00
yindo commented 2026-02-22 18:25:04 -05:00
Author
Owner
Copy Link
Copy Source

@daaddd commented on GitHub (Sep 12, 2024):

Screenshot 2024-09-12 at 11 05 07 PM Screenshot 2024-09-12 at 11 05 23 PM
@daaddd commented on GitHub (Sep 12, 2024): <img width="1002" alt="Screenshot 2024-09-12 at 11 05 07 PM" src="https://github.com/user-attachments/assets/bcfb3d85-d5f9-4c4e-ad60-0adb20d385ea"> <img width="1721" alt="Screenshot 2024-09-12 at 11 05 23 PM" src="https://github.com/user-attachments/assets/6dcd2fb3-8cd7-4e55-a650-d4f73f935f50">
yindo commented 2026-02-22 18:25:04 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 13, 2024):

Closing as this is a config/support issue (not a bug yet!). Anyways, things to look into:

  • GPT-4o is usually very good with tools, so that should not be an issue in this specific instance
  • When you invoke @agent do you see the container logs say something like [AgentHandler] Loading <custom-tool>. This would help us know if the tool is even being loaded at all.
  • Does the plugin.json for schedule-google-calendar-event exist in a folder with the _exact same name?
    --- /plugins/agent-skill/schedule-google-calendar-event
    -----------------------plugin.json
    -----------------------handle.js

Otherwise, everything looks good to me

@timothycarambat commented on GitHub (Sep 13, 2024): Closing as this is a config/support issue (not a bug yet!). Anyways, things to look into: - GPT-4o is usually very good with tools, so that should not be an issue in this specific instance - When you invoke `@agent` do you see the container logs say something like `[AgentHandler] Loading <custom-tool>`. This would help us know if the tool is even being loaded at all. - Does the `plugin.json` for `schedule-google-calendar-event` exist in a folder with the _exact same name? --- `/plugins/agent-skill/schedule-google-calendar-event` -----------------------`plugin.json` -----------------------`handle.js` Otherwise, everything looks good to me
yindo commented 2026-02-22 18:25:04 -05:00
Author
Owner
Copy Link
Copy Source

@daaddd commented on GitHub (Sep 13, 2024):

Yes I do see that but no invites are sent , only when invoked in chat directly:

[backend] info: [CollectorApi] Document created successfully. It is now available in documents.
[backend] info: [TELEMETRY SENT] {"event":"raw_document_uploaded","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}}
[backend] info: [Event Logged] - api_raw_document_uploaded
[backend] info: [TELEMETRY SENT] {"event":"workspace_thread_created","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"multiUserMode":true,"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}}
[backend] info: [Event Logged] - api_workspace_thread_created
[backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}}
[backend] info: [Event Logged] - api_sent_chat
[backend] info: [TELEMETRY SENT] {"event":"agent_chat_started","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}}
[backend] info: [EphemeralAgentHandler] Start fa44b952-3dec-40ee-84ba-08d613ceaf5c::openai:gpt-4o
[backend] info: [EphemeralAgentHandler] Attached httpSocket plugin to Agent cluster
[backend] info: [EphemeralAgentHandler] Attaching user and default agent to Agent cluster.
[backend] info: [EphemeralAgentHandler] Attached rag-memory plugin to Agent cluster
[backend] info: [EphemeralAgentHandler] Attached document-summarizer plugin to Agent cluster
[backend] info: [EphemeralAgentHandler] Attached web-scraping plugin to Agent cluster
[backend] info: [TELEMETRY SENT] {"event":"agent_chat_sent","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}}
[backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}}
[backend] info: [Event Logged] - api_sent_chat
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[collector] info: -- Working Raw Text doc Memory for thread c2f8836d-c79c-4b11-9e9d-e4976efead61 --
[collector] info: [SUCCESS]: Raw text and metadata saved & ready for embedding.

[backend] info: [CollectorApi] Document created successfully. It is now available in documents.
[backend] info: [TELEMETRY SENT] {"event":"raw_document_uploaded","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}}
[backend] info: [Event Logged] - api_raw_document_uploaded
[backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}}
[backend] info: [Event Logged] - api_sent_chat
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[collector] info: -- Working Raw Text doc Memory for thread c2f8836d-c79c-4b11-9e9d-e4976efead61 --
[collector] info: [SUCCESS]: Raw text and metadata saved & ready for embedding.

[backend] info: [CollectorApi] Document created successfully. It is now available in documents.
[backend] info: [TELEMETRY SENT] {"event":"raw_document_uploaded","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}}
[backend] info: [Event Logged] - api_raw_document_uploaded
@daaddd commented on GitHub (Sep 13, 2024): Yes I do see that but no invites are sent , only when invoked in chat directly: ``` [backend] info: [CollectorApi] Document created successfully. It is now available in documents. [backend] info: [TELEMETRY SENT] {"event":"raw_document_uploaded","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}} [backend] info: [Event Logged] - api_raw_document_uploaded [backend] info: [TELEMETRY SENT] {"event":"workspace_thread_created","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"multiUserMode":true,"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}} [backend] info: [Event Logged] - api_workspace_thread_created [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}} [backend] info: [Event Logged] - api_sent_chat [backend] info: [TELEMETRY SENT] {"event":"agent_chat_started","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}} [backend] info: [EphemeralAgentHandler] Start fa44b952-3dec-40ee-84ba-08d613ceaf5c::openai:gpt-4o [backend] info: [EphemeralAgentHandler] Attached httpSocket plugin to Agent cluster [backend] info: [EphemeralAgentHandler] Attaching user and default agent to Agent cluster. [backend] info: [EphemeralAgentHandler] Attached rag-memory plugin to Agent cluster [backend] info: [EphemeralAgentHandler] Attached document-summarizer plugin to Agent cluster [backend] info: [EphemeralAgentHandler] Attached web-scraping plugin to Agent cluster [backend] info: [TELEMETRY SENT] {"event":"agent_chat_sent","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}} [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}} [backend] info: [Event Logged] - api_sent_chat [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [collector] info: -- Working Raw Text doc Memory for thread c2f8836d-c79c-4b11-9e9d-e4976efead61 -- [collector] info: [SUCCESS]: Raw text and metadata saved & ready for embedding. [backend] info: [CollectorApi] Document created successfully. It is now available in documents. [backend] info: [TELEMETRY SENT] {"event":"raw_document_uploaded","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}} [backend] info: [Event Logged] - api_raw_document_uploaded [backend] info: [TELEMETRY SENT] {"event":"sent_chat","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"LLMSelection":"openai","Embedder":"openai","VectorDbSelection":"lancedb","TTSSelection":"native","runtime":"docker"}} [backend] info: [Event Logged] - api_sent_chat [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [collector] info: -- Working Raw Text doc Memory for thread c2f8836d-c79c-4b11-9e9d-e4976efead61 -- [collector] info: [SUCCESS]: Raw text and metadata saved & ready for embedding. [backend] info: [CollectorApi] Document created successfully. It is now available in documents. [backend] info: [TELEMETRY SENT] {"event":"raw_document_uploaded","distinctId":"7be12400-3dfc-45f7-9e33-4a883b1dfd53","properties":{"runtime":"docker"}} [backend] info: [Event Logged] - api_raw_document_uploaded ```
yindo commented 2026-02-22 18:25:04 -05:00
Author
Owner
Copy Link
Copy Source

@daaddd commented on GitHub (Sep 13, 2024):

Does the plugin.json for schedule-google-calendar-event exist in a folder with the _exact same name?

Yes, all good there, since it works when calling directly from the workspace chat

@daaddd commented on GitHub (Sep 13, 2024): Does the plugin.json for schedule-google-calendar-event exist in a folder with the _exact same name? Yes, all good there, since it works when calling directly from the workspace chat
yindo commented 2026-02-22 18:25:04 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 13, 2024):

EphemeralAgentHandler is when called via the API

I totally overlooked that detail. We dont have custom plugins integrated into the API agent caller yet.

https://github.com/Mintplex-Labs/anything-llm/blob/297b8aaf01aa16329ed299b7b02fe5efcb72af39/server/utils/agents/ephemeral.js#L129

Let me update this issue and work on that real quick.

@timothycarambat commented on GitHub (Sep 13, 2024): `EphemeralAgentHandler` is when called via the API I totally overlooked that detail. We dont have custom plugins integrated into the API agent caller yet. https://github.com/Mintplex-Labs/anything-llm/blob/297b8aaf01aa16329ed299b7b02fe5efcb72af39/server/utils/agents/ephemeral.js#L129 Let me update this issue and work on that real quick.
yindo commented 2026-02-22 18:25:04 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 13, 2024):

Building here: https://github.com/Mintplex-Labs/anything-llm/actions/runs/10856446593/job/30131093448

@timothycarambat commented on GitHub (Sep 13, 2024): Building here: https://github.com/Mintplex-Labs/anything-llm/actions/runs/10856446593/job/30131093448
yindo commented 2026-02-22 18:25:05 -05:00
Author
Owner
Copy Link
Copy Source

@daaddd commented on GitHub (Sep 13, 2024):

@timothycarambat is the image already available to be used in docker? thanks a ton for working on this so quickly!

@daaddd commented on GitHub (Sep 13, 2024): @timothycarambat is the image already available to be used in docker? thanks a ton for working on this so quickly!
yindo commented 2026-02-22 18:25:05 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 13, 2024):

@timothycarambat is the image already available to be used in docker? thanks a ton for working on this so quickly!

Yes, build looks like it was completed and pushed. Just repull latest image and you are good to go

@timothycarambat commented on GitHub (Sep 13, 2024): > @timothycarambat is the image already available to be used in docker? thanks a ton for working on this so quickly! Yes, build looks like it was completed and pushed. Just repull latest image and you are good to go
yindo changed title from [Feat]: Custom skill calling via API agent to [GH-ISSUE #2278] [Feat]: Custom skill calling via API agent 2026-06-05 14:41:01 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1487
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?