Model 'llama3' Not Found but from another Docker container #104

New Issue

2026-02-15T16:28:01-05:00

yindo commented

2026-02-15 16:28:01 -05:00

Originally created by @bauerem on GitHub (Jun 22, 2024).

If I make a request to the ollama-service with http://localhost:11434 then I get a correct response.
If I make a request from another Docker container:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import requests
import json
from ollama import Client
client = Client(host='http://ollama-service:11434')

app = FastAPI()

class UserInput(BaseModel):
    input: str

@app.post("/process")
def process_input(user_input: UserInput):
    model_name = "llama3"
    messages = [
        {
            'role': 'user',
            'content': user_input.input,
        },
    ]
    
    try:
        response = client.chat(model=model_name, messages=messages)
        
        result = response['message']['content']
        
        return {"answer": result}
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Error processing the request with Ollama: {str(e)}")

I get the error:
model 'llama3' not found, try pulling it first

If I run this FastAPI app outside the Docker container and replace the 'ollama-service' with 'localhost', everything works fine.

Docker compose file

 version: '3.8'

services:
  fastapi-app:
    build: ./api
    volumes:
      - .:/usr/src/app
    ports:
      - "8000:8000"
    depends_on:
      - ollama-service

  ollama-service:
    image: ollama/ollama:latest 
    environment:
      - OLLAMA_MODEL=llama3 
    ports:
      - "11434:11434"

Dockerfile

FROM python:3.10

WORKDIR /app

COPY . /app

RUN pip install --no-cache-dir -r requirements.txt

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Originally created by @bauerem on GitHub (Jun 22, 2024). If I make a request to the ollama-service with `http://localhost:11434` then I get a correct response. If I make a request from another Docker container: from fastapi import FastAPI, HTTPException from pydantic import BaseModel import requests import json from ollama import Client client = Client(host='http://ollama-service:11434') app = FastAPI() class UserInput(BaseModel): input: str @app.post("/process") def process_input(user_input: UserInput): model_name = "llama3" messages = [ { 'role': 'user', 'content': user_input.input, }, ] try: response = client.chat(model=model_name, messages=messages) result = response['message']['content'] return {"answer": result} except Exception as e: raise HTTPException(status_code=500, detail=f"Error processing the request with Ollama: {str(e)}") I get the error: ` model 'llama3' not found, try pulling it first` If I run this FastAPI app outside the Docker container and replace the 'ollama-service' with 'localhost', everything works fine. **Docker compose file** version: '3.8' services: fastapi-app: build: ./api volumes: - .:/usr/src/app ports: - "8000:8000" depends_on: - ollama-service ollama-service: image: ollama/ollama:latest environment: - OLLAMA_MODEL=llama3 ports: - "11434:11434" **Dockerfile** FROM python:3.10 WORKDIR /app COPY . /app RUN pip install --no-cache-dir -r requirements.txt EXPOSE 8000 CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

yindo closed this issue

2026-02-15 16:28:01 -05:00

yindo referenced this issue

2026-02-15 16:30:05 -05:00

[PR #104] [MERGED] Bump actions/upload-artifact from 3 to 4 #357

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#104