[PR #194] [MERGED] Use native system instructions for Gemini 1.5 models #346

New Issue

2026-02-15T20:15:12-05:00

yindo commented

2026-02-15 20:15:12 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/pipelines/pull/194
Author: @rotemdan
Created: 8/2/2024
Status: ✅ Merged
Merged: 8/2/2024
Merged by: @justinh-rahb

Base: main ← Head: use-native-system-message-for-gemini-1.5

📝 Commits (2)

a94512d Use native system instructions for Gemini 1.5 models
aabf46c Increment version to 1.3

📊 Changes

1 file changed (+9 additions, -6 deletions)

View changed files

📝 examples/pipelines/providers/google_manifold_pipeline.py (+9 -6)

📄 Description

The current code in google_manifold_pipeline adds a "system"-like prompt by prepending the request with a message containing the system prompt:

if system_message:
    contents.insert(0, {"role": "user", "parts": [{"text": f"System: {system_message}"}]})

On Gemini 1.5 models, Google added support for native system instructions. In the Python API it is passed as an argument for the constructor:

model = genai.GenerativeModel(model_name=model_id, system_instruction=system_message)

The equivalent in the REST API is documented here:

curl https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$GOOGLE_API_KEY \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
    "parts":
      { "text": "You are Neko the cat respond like one"}},
    "contents": {
      "parts": {
        "text": "Good morning! How are you?"}}}'

In my patch I added support for it when the string gemini-1.5 is part of the model id:

            if "gemini-1.5" in model_id:
                model = genai.GenerativeModel(model_name=model_id, system_instruction=system_message)
            else:
                if system_message:
                    contents.insert(0, {"role": "user", "parts": [{"text": f"System: {system_message}"}]})
                
                model = genai.GenerativeModel(model_name=model_id)

Gemini 1.0 and Gemma models don't support system instructions, so they will fall back to the other approach. Note: after trying to use system_instruction with Gemini 1.0 I got an error from the server, but both Gemma models (gemma2-9b-it and gemma-7b-it) did accept the instruction without error and showed knowledge of it. Officially they don't support system instructions, so it could be a workaround prompt on the server.

Future models, say, like gemini-2.0 will need to be added manually when they become available on the API, otherwise they will fall back to the other approach.

The information returned by the API unfortunately doesn't indicate if the model supports system instructions. Here is for example, the information for Gemini 1.5 pro (returned via https://generativelanguage.googleapis.com/v1beta/models):

   {
      "name": "models/gemini-1.5-pro",
      "version": "001",
      "displayName": "Gemini 1.5 Pro",
      "description": "Mid-size multimodal model that supports up to 2 million tokens",
      "inputTokenLimit": 2097152,
      "outputTokenLimit": 8192,
      "supportedGenerationMethods": [
        "generateContent",
        "countTokens"
      ],
      "temperature": 1,
      "topP": 0.95,
      "topK": 64,
      "maxTemperature": 2
    },

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/pipelines/pull/194 **Author:** [@rotemdan](https://github.com/rotemdan) **Created:** 8/2/2024 **Status:** ✅ Merged **Merged:** 8/2/2024 **Merged by:** [@justinh-rahb](https://github.com/justinh-rahb) **Base:** `main` ← **Head:** `use-native-system-message-for-gemini-1.5` --- ### 📝 Commits (2) - [`a94512d`](https://github.com/open-webui/pipelines/commit/a94512dad42716bf2f1c391fe930605302c173af) Use native system instructions for Gemini 1.5 models - [`aabf46c`](https://github.com/open-webui/pipelines/commit/aabf46c8e9ca230b87681e53d122c25b4c41c94b) Increment version to 1.3 ### 📊 Changes **1 file changed** (+9 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `examples/pipelines/providers/google_manifold_pipeline.py` (+9 -6) </details> ### 📄 Description The current code in `google_manifold_pipeline` adds a "system"-like prompt by prepending the request with a message containing the system prompt: ```py if system_message: contents.insert(0, {"role": "user", "parts": [{"text": f"System: {system_message}"}]}) ``` On Gemini 1.5 models, Google added support for native system instructions. In the Python API it is passed as an argument for the constructor: ```py model = genai.GenerativeModel(model_name=model_id, system_instruction=system_message) ``` The equivalent in the REST API is documented [here](https://ai.google.dev/gemini-api/docs/system-instructions?lang=rest): ``` curl https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$GOOGLE_API_KEY \ -H 'Content-Type: application/json' \ -d '{ "system_instruction": { "parts": { "text": "You are Neko the cat respond like one"}}, "contents": { "parts": { "text": "Good morning! How are you?"}}}' ``` In my patch I added support for it when the string `gemini-1.5` is part of the model id: ```py if "gemini-1.5" in model_id: model = genai.GenerativeModel(model_name=model_id, system_instruction=system_message) else: if system_message: contents.insert(0, {"role": "user", "parts": [{"text": f"System: {system_message}"}]}) model = genai.GenerativeModel(model_name=model_id) ``` Gemini 1.0 and Gemma models don't support system instructions, so they will fall back to the other approach. **Note**: after trying to use `system_instruction` with Gemini 1.0 I got an error from the server, but both Gemma models (`gemma2-9b-it` and `gemma-7b-it`) did accept the instruction without error and showed knowledge of it. Officially they [don't support system instructions](https://ai.google.dev/gemma/docs/formatting#system-instructions), so it could be a workaround prompt on the server. Future models, say, like `gemini-2.0` will need to be added manually when they become available on the API, otherwise they will fall back to the other approach. The information returned by the API unfortunately doesn't indicate if the model supports system instructions. Here is for example, the information for Gemini 1.5 pro (returned via `https://generativelanguage.googleapis.com/v1beta/models`): ```json { "name": "models/gemini-1.5-pro", "version": "001", "displayName": "Gemini 1.5 Pro", "description": "Mid-size multimodal model that supports up to 2 million tokens", "inputTokenLimit": 2097152, "outputTokenLimit": 8192, "supportedGenerationMethods": [ "generateContent", "countTokens" ], "temperature": 1, "topP": 0.95, "topK": 64, "maxTemperature": 2 }, ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

yindo added the pull-request label 2026-02-15 20:15:12 -05:00

yindo closed this issue

2026-02-15 20:15:12 -05:00

yindo referenced this issue

2026-02-15 20:15:22 -05:00

[PR #346] [CLOSED] Some fixes to text_to_sql_pipeline.py #386

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: open-webui/pipelines#346