Fixed: Anthropic API Integration Update for System Message Handling #64

New Issue

Closed

opened 2026-02-15 19:15:19 -05:00 by yindo · 5 comments

yindo commented 2026-02-15 19:15:19 -05:00

Owner

Copy Link

Copy Source

Originally created by @petunder on GitHub (Jun 26, 2024).

Issue:

The previous version of the code encountered a problem with handling system messages when using the Anthropic API. System messages were not being correctly passed in API requests, which could lead to improper functioning of chat models.

Solution:

The following key changes were made to address the issue:

1. Updated the handling of system messages in the stream_response and get_completion methods.
2. Modified the format for passing messages to the Anthropic API.
3. Added additional error handling and debug information.

Change Details

1. System Message Handling

In both stream_response and get_completion methods, extraction of system message content was added:

system_message, messages = pop_system_message(messages)
system_message_content = system_message.get('content', '')

2. Message Format Modification

In the get_completion method, the way messages are formatted for the Anthropic API was changed:

formatted_messages = []
formatted_messages.extend([{"role": msg["role"], "content": msg["content"]} for msg in messages])

3. API Call Update

The Anthropic API call was updated in both methods. Now the system message is passed as a separate parameter:

response = self.client.messages.create(
    model=model_id,
    system=system_message_content,
    messages=formatted_messages,
    # ... other parameters ...
)

4. Debug Information Addition

Debug outputs were added to facilitate problem diagnosis:

print("Formatted messages:", formatted_messages)
print("Model ID:", model_id)

5. Improved Error Handling

More detailed handling of BadRequestError was added:

except anthropic.BadRequestError as e:
    print(f"Bad Request Error: {e}")
    print(f"Request data: {e.request}")
    print(f"Response data: {e.response}")
    raise

It works. I cannot edit https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/anthropic_manifold_pipeline.py please change the code. Other users can update this pipe locally, here full new code (copy/paste):

(BUG FIX VERSION)

"""
title: Anthropic Manifold Pipeline
author: justinh-rahb + petr shkumatov
date: 2024-06-26
version: 1.2
license: MIT
description: A pipeline for generating text using the Anthropic API.
requirements: requests, anthropic
environment_variables: ANTHROPIC_API_KEY
"""

import os
import anthropic
from anthropic import Anthropic, RateLimitError, APIStatusError, APIConnectionError
from anthropic.types import MessageParam
from schemas import OpenAIChatMessage
from typing import List, Union, Generator, Iterator
from pydantic import BaseModel
import requests

from utils.pipelines.main import pop_system_message


class Pipeline:
    class Valves(BaseModel):
        ANTHROPIC_API_KEY: str = ""

    def __init__(self):
        self.type = "manifold"
        self.id = "anthropic"
        self.name = "anthropic/"

        self.valves = self.Valves(
            **{"ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY", "")}
        )
        self.client = Anthropic(api_key="")

    def get_anthropic_models(self):
        # In the future, this could fetch models dynamically from Anthropic
        return [
            {"id": "claude-3-haiku-20240307", "name": "claude-3-haiku"},
            {"id": "claude-3-opus-20240229", "name": "claude-3-opus"},
            {"id": "claude-3-sonnet-20240229", "name": "claude-3-sonnet"},
            {"id": "claude-3-5-sonnet-20240620", "name": "claude-3.5-sonnet"},
            # Add other Anthropic models here as they become available
        ]

    async def on_startup(self):
        print(f"on_startup:{__name__}")
        pass

    async def on_shutdown(self):
        print(f"on_shutdown:{__name__}")
        pass

    async def on_valves_updated(self):
        # This function is called when the valves are updated.
        self.client = Anthropic(api_key=self.valves.ANTHROPIC_API_KEY)
        pass

    # Pipelines are the models that are available in the manifold.
    # It can be a list or a function that returns a list.
    def pipelines(self) -> List[dict]:
        return self.get_anthropic_models()

    def pipe(
        self, user_message: str, model_id: str, messages: List[dict], body: dict
    ) -> Union[str, Generator, Iterator]:
        try:
            if "user" in body:
                del body["user"]
            if "chat_id" in body:
                del body["chat_id"]
            if "title" in body:
                del body["title"]

            if body.get("stream", False):
                return self.stream_response(model_id, messages, body)
            else:
                return self.get_completion(model_id, messages, body)
        except (RateLimitError, APIStatusError, APIConnectionError) as e:
            return f"Error: {e}"

    def stream_response(
        self, model_id: str, messages: List[dict], body: dict
    ) -> Generator:
        system_message, messages = pop_system_message(messages)

        
        params = {
            "model": model_id,
            "messages": messages,
            "max_tokens": body.get("max_tokens", 4096),
            "temperature": body.get("temperature", 0.8),
            "top_k": body.get("top_k", 40),
            "top_p": body.get("top_p", 0.9),
            "stop_sequences": body.get("stop", []),
            "stream": True
        }

        if system_message:
            system_message_content = system_message.get('content', '')
            params["system"] = system_message_content

        print(f"stream_response messages: {messages}")
        print(f"stream_response system_message: {system_message}")

        stream = self.client.messages.create(**params)

        for chunk in stream:
            if chunk.type in ("content_block_start", "content_block_delta"):
                yield chunk.content_block.text if chunk.type == "content_block_start" else chunk.delta.text

    def get_completion(self, model_id: str, messages: List[dict], body: dict) -> str:
        system_message, messages = pop_system_message(messages)
        
        formatted_messages = [{"role": msg["role"], "content": msg["content"]} for msg in messages]
        
        params = {
            "model": model_id,
            "messages": formatted_messages,
            "max_tokens": body.get("max_tokens", 4096),
            "temperature": body.get("temperature", 0.8),
            "top_k": body.get("top_k", 40),
            "top_p": body.get("top_p", 0.9),
            "stop_sequences": body.get("stop", [])
        }
        
        #fix NULL system message
        if system_message: 
            system_message_content = system_message.get('content', '')
            params["system"] = system_message_content

        print("Formatted messages:", formatted_messages)
        print("Model ID:", model_id)

        try:
            response = self.client.messages.create(**params)
            return response.content[0].text
        except anthropic.BadRequestError as e:
            print(f"Bad Request Error: {e}")
            print(f"Request data: {e.request}")
            print(f"Response data: {e.response}")
            raise

Originally created by @petunder on GitHub (Jun 26, 2024). Issue: The previous version of the code encountered a problem with handling system messages when using the Anthropic API. System messages were not being correctly passed in API requests, which could lead to improper functioning of chat models. Solution: The following key changes were made to address the issue: 1. Updated the handling of system messages in the `stream_response` and `get_completion` methods. 2. Modified the format for passing messages to the Anthropic API. 3. Added additional error handling and debug information. Change Details 1. System Message Handling In both `stream_response` and `get_completion` methods, extraction of system message content was added: ```python system_message, messages = pop_system_message(messages) system_message_content = system_message.get('content', '') ``` 2. Message Format Modification In the `get_completion` method, the way messages are formatted for the Anthropic API was changed: ```python formatted_messages = [] formatted_messages.extend([{"role": msg["role"], "content": msg["content"]} for msg in messages]) ``` 3. API Call Update The Anthropic API call was updated in both methods. Now the system message is passed as a separate parameter: ```python response = self.client.messages.create( model=model_id, system=system_message_content, messages=formatted_messages, # ... other parameters ... ) ``` 4. Debug Information Addition Debug outputs were added to facilitate problem diagnosis: ```python print("Formatted messages:", formatted_messages) print("Model ID:", model_id) ``` 5. Improved Error Handling More detailed handling of BadRequestError was added: ```python except anthropic.BadRequestError as e: print(f"Bad Request Error: {e}") print(f"Request data: {e.request}") print(f"Response data: {e.response}") raise ``` It works. I cannot edit https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/anthropic_manifold_pipeline.py please change the code. Other users can update this pipe locally, here full new code (copy/paste): (BUG FIX VERSION) ```python """ title: Anthropic Manifold Pipeline author: justinh-rahb + petr shkumatov date: 2024-06-26 version: 1.2 license: MIT description: A pipeline for generating text using the Anthropic API. requirements: requests, anthropic environment_variables: ANTHROPIC_API_KEY """ import os import anthropic from anthropic import Anthropic, RateLimitError, APIStatusError, APIConnectionError from anthropic.types import MessageParam from schemas import OpenAIChatMessage from typing import List, Union, Generator, Iterator from pydantic import BaseModel import requests from utils.pipelines.main import pop_system_message class Pipeline: class Valves(BaseModel): ANTHROPIC_API_KEY: str = "" def __init__(self): self.type = "manifold" self.id = "anthropic" self.name = "anthropic/" self.valves = self.Valves( **{"ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY", "")} ) self.client = Anthropic(api_key="") def get_anthropic_models(self): # In the future, this could fetch models dynamically from Anthropic return [ {"id": "claude-3-haiku-20240307", "name": "claude-3-haiku"}, {"id": "claude-3-opus-20240229", "name": "claude-3-opus"}, {"id": "claude-3-sonnet-20240229", "name": "claude-3-sonnet"}, {"id": "claude-3-5-sonnet-20240620", "name": "claude-3.5-sonnet"}, # Add other Anthropic models here as they become available ] async def on_startup(self): print(f"on_startup:{__name__}") pass async def on_shutdown(self): print(f"on_shutdown:{__name__}") pass async def on_valves_updated(self): # This function is called when the valves are updated. self.client = Anthropic(api_key=self.valves.ANTHROPIC_API_KEY) pass # Pipelines are the models that are available in the manifold. # It can be a list or a function that returns a list. def pipelines(self) -> List[dict]: return self.get_anthropic_models() def pipe( self, user_message: str, model_id: str, messages: List[dict], body: dict ) -> Union[str, Generator, Iterator]: try: if "user" in body: del body["user"] if "chat_id" in body: del body["chat_id"] if "title" in body: del body["title"] if body.get("stream", False): return self.stream_response(model_id, messages, body) else: return self.get_completion(model_id, messages, body) except (RateLimitError, APIStatusError, APIConnectionError) as e: return f"Error: {e}" def stream_response( self, model_id: str, messages: List[dict], body: dict ) -> Generator: system_message, messages = pop_system_message(messages) params = { "model": model_id, "messages": messages, "max_tokens": body.get("max_tokens", 4096), "temperature": body.get("temperature", 0.8), "top_k": body.get("top_k", 40), "top_p": body.get("top_p", 0.9), "stop_sequences": body.get("stop", []), "stream": True } if system_message: system_message_content = system_message.get('content', '') params["system"] = system_message_content print(f"stream_response messages: {messages}") print(f"stream_response system_message: {system_message}") stream = self.client.messages.create(**params) for chunk in stream: if chunk.type in ("content_block_start", "content_block_delta"): yield chunk.content_block.text if chunk.type == "content_block_start" else chunk.delta.text def get_completion(self, model_id: str, messages: List[dict], body: dict) -> str: system_message, messages = pop_system_message(messages) formatted_messages = [{"role": msg["role"], "content": msg["content"]} for msg in messages] params = { "model": model_id, "messages": formatted_messages, "max_tokens": body.get("max_tokens", 4096), "temperature": body.get("temperature", 0.8), "top_k": body.get("top_k", 40), "top_p": body.get("top_p", 0.9), "stop_sequences": body.get("stop", []) } #fix NULL system message if system_message: system_message_content = system_message.get('content', '') params["system"] = system_message_content print("Formatted messages:", formatted_messages) print("Model ID:", model_id) try: response = self.client.messages.create(**params) return response.content[0].text except anthropic.BadRequestError as e: print(f"Bad Request Error: {e}") print(f"Request data: {e.request}") print(f"Response data: {e.response}") raise ```

yindo closed this issue 2026-02-15 19:15:19 -05:00

yindo commented 2026-02-15 19:15:20 -05:00

Author

Owner

Copy Link

Copy Source

@justinh-rahb commented on GitHub (Jun 26, 2024):

Hi! Thanks for your efforts in fixing this. As it happens I've already been working on a port of this original code to the new function platform and resolved this issue. My backport is largely complete and my system message handling is similar to how you've done here, so I'll add you as a coauthor to the PR. The backport also will handle images for Claude 3.5 Sonnet.

@justinh-rahb commented on GitHub (Jun 26, 2024): Hi! Thanks for your efforts in fixing this. As it happens I've already been working on a port of this original code to the new function platform and resolved this issue. My backport is largely complete and my system message handling is similar to how you've done here, so I'll add you as a coauthor to the PR. The backport also will handle images for Claude 3.5 Sonnet.

yindo commented 2026-02-15 19:15:20 -05:00

Author

Owner

Copy Link

Copy Source

@justinh-rahb commented on GitHub (Jun 26, 2024):

Fixed:

• Pipeline overriding two simultaneos requests. (#124)

@petunder I'll need your email address to add you as a coauthor.

@justinh-rahb commented on GitHub (Jun 26, 2024): Fixed: - #124 @petunder I'll need your email address to add you as a coauthor.

yindo commented 2026-02-15 19:15:20 -05:00

Author

Owner

Copy Link

Copy Source

@petunder commented on GitHub (Jun 26, 2024):

Sorry, first version was a bug. This code works well with or without system prompt:

"""
title: Anthropic Manifold Pipeline
author: justinh-rahb + petr shkumatov
date: 2024-06-26
version: 1.2
license: MIT
description: A pipeline for generating text using the Anthropic API.
requirements: requests, anthropic
environment_variables: ANTHROPIC_API_KEY
"""

import os
import anthropic
from anthropic import Anthropic, RateLimitError, APIStatusError, APIConnectionError
from anthropic.types import MessageParam
from schemas import OpenAIChatMessage
from typing import List, Union, Generator, Iterator
from pydantic import BaseModel
import requests

from utils.pipelines.main import pop_system_message


class Pipeline:
    class Valves(BaseModel):
        ANTHROPIC_API_KEY: str = ""

    def __init__(self):
        self.type = "manifold"
        self.id = "anthropic"
        self.name = "anthropic/"

        self.valves = self.Valves(
            **{"ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY", "")}
        )
        self.client = Anthropic(api_key="")

    def get_anthropic_models(self):
        # In the future, this could fetch models dynamically from Anthropic
        return [
            {"id": "claude-3-haiku-20240307", "name": "claude-3-haiku"},
            {"id": "claude-3-opus-20240229", "name": "claude-3-opus"},
            {"id": "claude-3-sonnet-20240229", "name": "claude-3-sonnet"},
            {"id": "claude-3-5-sonnet-20240620", "name": "claude-3.5-sonnet"},
            # Add other Anthropic models here as they become available
        ]

    async def on_startup(self):
        print(f"on_startup:{__name__}")
        pass

    async def on_shutdown(self):
        print(f"on_shutdown:{__name__}")
        pass

    async def on_valves_updated(self):
        # This function is called when the valves are updated.
        self.client = Anthropic(api_key=self.valves.ANTHROPIC_API_KEY)
        pass

    # Pipelines are the models that are available in the manifold.
    # It can be a list or a function that returns a list.
    def pipelines(self) -> List[dict]:
        return self.get_anthropic_models()

    def pipe(
        self, user_message: str, model_id: str, messages: List[dict], body: dict
    ) -> Union[str, Generator, Iterator]:
        try:
            if "user" in body:
                del body["user"]
            if "chat_id" in body:
                del body["chat_id"]
            if "title" in body:
                del body["title"]

            if body.get("stream", False):
                return self.stream_response(model_id, messages, body)
            else:
                return self.get_completion(model_id, messages, body)
        except (RateLimitError, APIStatusError, APIConnectionError) as e:
            return f"Error: {e}"

    def stream_response(
        self, model_id: str, messages: List[dict], body: dict
    ) -> Generator:
        system_message, messages = pop_system_message(messages)

        
        params = {
            "model": model_id,
            "messages": messages,
            "max_tokens": body.get("max_tokens", 4096),
            "temperature": body.get("temperature", 0.8),
            "top_k": body.get("top_k", 40),
            "top_p": body.get("top_p", 0.9),
            "stop_sequences": body.get("stop", []),
            "stream": True
        }

        if system_message:
            system_message_content = system_message.get('content', '')
            params["system"] = system_message_content

        print(f"stream_response messages: {messages}")
        print(f"stream_response system_message: {system_message}")

        stream = self.client.messages.create(**params)

        for chunk in stream:
            if chunk.type in ("content_block_start", "content_block_delta"):
                yield chunk.content_block.text if chunk.type == "content_block_start" else chunk.delta.text

    def get_completion(self, model_id: str, messages: List[dict], body: dict) -> str:
        system_message, messages = pop_system_message(messages)
        
        formatted_messages = [{"role": msg["role"], "content": msg["content"]} for msg in messages]
        
        params = {
            "model": model_id,
            "messages": formatted_messages,
            "max_tokens": body.get("max_tokens", 4096),
            "temperature": body.get("temperature", 0.8),
            "top_k": body.get("top_k", 40),
            "top_p": body.get("top_p", 0.9),
            "stop_sequences": body.get("stop", [])
        }
        
        #fix NULL system message
        if system_message: 
            system_message_content = system_message.get('content', '')
            params["system"] = system_message_content

        print("Formatted messages:", formatted_messages)
        print("Model ID:", model_id)

        try:
            response = self.client.messages.create(**params)
            return response.content[0].text
        except anthropic.BadRequestError as e:
            print(f"Bad Request Error: {e}")
            print(f"Request data: {e.request}")
            print(f"Response data: {e.response}")
            raise

@petunder commented on GitHub (Jun 26, 2024): Sorry, first version was a bug. This code works well with or without system prompt: ```python """ title: Anthropic Manifold Pipeline author: justinh-rahb + petr shkumatov date: 2024-06-26 version: 1.2 license: MIT description: A pipeline for generating text using the Anthropic API. requirements: requests, anthropic environment_variables: ANTHROPIC_API_KEY """ import os import anthropic from anthropic import Anthropic, RateLimitError, APIStatusError, APIConnectionError from anthropic.types import MessageParam from schemas import OpenAIChatMessage from typing import List, Union, Generator, Iterator from pydantic import BaseModel import requests from utils.pipelines.main import pop_system_message class Pipeline: class Valves(BaseModel): ANTHROPIC_API_KEY: str = "" def __init__(self): self.type = "manifold" self.id = "anthropic" self.name = "anthropic/" self.valves = self.Valves( **{"ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY", "")} ) self.client = Anthropic(api_key="") def get_anthropic_models(self): # In the future, this could fetch models dynamically from Anthropic return [ {"id": "claude-3-haiku-20240307", "name": "claude-3-haiku"}, {"id": "claude-3-opus-20240229", "name": "claude-3-opus"}, {"id": "claude-3-sonnet-20240229", "name": "claude-3-sonnet"}, {"id": "claude-3-5-sonnet-20240620", "name": "claude-3.5-sonnet"}, # Add other Anthropic models here as they become available ] async def on_startup(self): print(f"on_startup:{__name__}") pass async def on_shutdown(self): print(f"on_shutdown:{__name__}") pass async def on_valves_updated(self): # This function is called when the valves are updated. self.client = Anthropic(api_key=self.valves.ANTHROPIC_API_KEY) pass # Pipelines are the models that are available in the manifold. # It can be a list or a function that returns a list. def pipelines(self) -> List[dict]: return self.get_anthropic_models() def pipe( self, user_message: str, model_id: str, messages: List[dict], body: dict ) -> Union[str, Generator, Iterator]: try: if "user" in body: del body["user"] if "chat_id" in body: del body["chat_id"] if "title" in body: del body["title"] if body.get("stream", False): return self.stream_response(model_id, messages, body) else: return self.get_completion(model_id, messages, body) except (RateLimitError, APIStatusError, APIConnectionError) as e: return f"Error: {e}" def stream_response( self, model_id: str, messages: List[dict], body: dict ) -> Generator: system_message, messages = pop_system_message(messages) params = { "model": model_id, "messages": messages, "max_tokens": body.get("max_tokens", 4096), "temperature": body.get("temperature", 0.8), "top_k": body.get("top_k", 40), "top_p": body.get("top_p", 0.9), "stop_sequences": body.get("stop", []), "stream": True } if system_message: system_message_content = system_message.get('content', '') params["system"] = system_message_content print(f"stream_response messages: {messages}") print(f"stream_response system_message: {system_message}") stream = self.client.messages.create(**params) for chunk in stream: if chunk.type in ("content_block_start", "content_block_delta"): yield chunk.content_block.text if chunk.type == "content_block_start" else chunk.delta.text def get_completion(self, model_id: str, messages: List[dict], body: dict) -> str: system_message, messages = pop_system_message(messages) formatted_messages = [{"role": msg["role"], "content": msg["content"]} for msg in messages] params = { "model": model_id, "messages": formatted_messages, "max_tokens": body.get("max_tokens", 4096), "temperature": body.get("temperature", 0.8), "top_k": body.get("top_k", 40), "top_p": body.get("top_p", 0.9), "stop_sequences": body.get("stop", []) } #fix NULL system message if system_message: system_message_content = system_message.get('content', '') params["system"] = system_message_content print("Formatted messages:", formatted_messages) print("Model ID:", model_id) try: response = self.client.messages.create(**params) return response.content[0].text except anthropic.BadRequestError as e: print(f"Bad Request Error: {e}") print(f"Request data: {e.request}") print(f"Response data: {e.response}") raise ```

yindo commented 2026-02-15 19:15:20 -05:00

Author

Owner

Copy Link

Copy Source

@petunder commented on GitHub (Jun 26, 2024):

Fixed:

• Update Anthropic Manifold Pipeline to v0.1.2 #124

@petunder I'll need your email address to add you as a coauthor.

petya.uz@tuta.io

Pls check new version. There is a bug fix

@petunder commented on GitHub (Jun 26, 2024): > Fixed: > > * [Update Anthropic Manifold Pipeline to v0.1.2 #124](https://github.com/open-webui/pipelines/pull/124) > > @petunder I'll need your email address to add you as a coauthor. petya.uz@tuta.io Pls check new version. There is a bug fix

yindo commented 2026-02-15 19:15:20 -05:00

Author

Owner

Copy Link

Copy Source

@justinh-rahb commented on GitHub (Jun 26, 2024):

@petunder afaik my current PR's code works, I don't think I'll be needing the above but feel free to open another PR after this is merged if you feel there's more improvements to make.

@justinh-rahb commented on GitHub (Jun 26, 2024): @petunder afaik my current PR's code works, I don't think I'll be needing the above but feel free to open another PR after this is merged if you feel there's more improvements to make.

yindo referenced this issue 2026-02-15 20:14:58 -05:00

[PR #64] [MERGED] Add ImageGen exaple (Using DALL-E) #291

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

gemini-permissive-safety

No results found.

Labels

Clear labels

bug

bug

enhancement

enhancement

good first issue

help wanted

pull-request

Mirrored from GitHub Pull Request

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

yindo

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: open-webui/pipelines#64