embed function with LLMs in Ollama #222

New Issue

2026-02-15T16:28:51-05:00

yindo commented

2026-02-15 16:28:51 -05:00

Originally created by @acraevschi on GitHub (Feb 18, 2025).

Hello everyone!

I recently discovered that Ollama has embed function. In this repo, there is an example of using it with 'llama3.2' (3B variant, by default). The output of this function is a vector of llama's hidden_size parameter (=3072).

Does this mean the model is doing more than just passing the input through the network (excluding the output layer) and instead applying some form of pooling?

Specifically, how is the final hidden representation being transformed from shape [input_tokenized_len, hidden_size] to [1, hidden_size]?

I understand how specialized sentence transformers achieve this, but in a typical decoder-only LLM like LLaMA, the hidden states usually retain the [input_tokenized_len, hidden_size] shape before the final output. Could someone clarify what transformation is applied here?

Originally created by @acraevschi on GitHub (Feb 18, 2025). Hello everyone! I recently discovered that Ollama has `embed` function. In this repo, there is an example of using it with `'llama3.2'` (3B variant, by default). The output of this function is a vector of llama's `hidden_size` parameter (=3072). Does this mean the model is doing more than just passing the input through the network (excluding the output layer) and instead applying some form of pooling? Specifically, how is the final hidden representation being transformed from shape `[input_tokenized_len, hidden_size]` to `[1, hidden_size]`? I understand how specialized sentence transformers achieve this, but in a typical decoder-only LLM like LLaMA, the hidden states usually retain the `[input_tokenized_len, hidden_size]` shape before the final output. Could someone clarify what transformation is applied here?

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: ollama/ollama-python#222

embed function with LLMs in Ollama #222

`embed` function with LLMs in Ollama #222